To provide an update on this RFC: @vortex73 has been working on a prototype for the ABI lowering library as part of GSoC. You can find the result here: https://github.com/llvm/llvm-project/pull/140112
This prototype implements:
- An ABI library with its own type system
- Mapping from Clang QualType to the ABI type system
- ABI calculation for the x86_64 SystemV (as well as BPF) targets, producing identical results to Clang’s existing code
- Conversion of the results back to Clang’s CGFunctionInfo format, so this slots into the existing code
I believe this prototype proves that the general approach is viable. It also shows that going through a separate ABI type system does not have any significant compile-time impact (it is <0.1% at -O0
even without going out of the way to optimize it).
I expect that extending this to other targets is going to require some additional extensions to the type system (e.g. x86_64, while probably the most complex ABI overall, does not need to deal with type unadjusted alignment, so it’s currently not part of the type system), but I think the core architecture of the prototype should otherwise be representative.
We plan to start upstreaming this work soon. Please let us know if there are any high level concerns.