Clang ABI library

Hello all,

There has apparently been discussion for some time now about creating an ABI support library in Clang to be used for external developers. For some time now I’ve been wondering about the status of the proposed ABI support library. But I realized that I’ve effectively already implemented a good part of it myself. As an external developer I’ve been building a project on top of Clang for some time now with the aim of implementing the following features:

  • Call into C and C++ functions declared in a header analyzed by Clang, including special members (including implicitly generated ones), operator overloads, those with defaulted arguments, and such,

  • Implement C and C++ functions declared in a header analyzed by Clang,

  • Access members of C and C++ types, including the virtual pointer and vtables,

  • Call C++ virtual functions, and implement types that effectively inherit from C++ types,

  • Code-generate C++ translation units into an LLVM module for use by external code generation mechanisms,

  • Create thunks with an arbitrary body for a given signature,

  • Perform C++ semantic analysis from external source; e.g. name lookup, template instantiation, overload resolution, and such,

  • Evaluate C++ macros and expressions within limited contexts; e.g. “Valid expression at global namespace scope in given TU”. I am aware of one other who would be looking for “Valid statements at function scope; for given imaginary function X with local variables Y and Z of type W.”,

  • Implement exception handling and RTTI with binary compatibility with Clang,

Some of this stuff has been easily accessible via existing APIs; but some has not. So far I’ve had success implementing most of these features most of the time, but I’ve reached the inescapable conclusion that some of them will require patches for Clang. Since I know that I’m not the only one who would like these features, then I may as well just upstream them.

For semantic analysis, parsing and macro evaluation, I’ve generally determined that the existing APIs offer all the features required, but the user-friendliness is often a bit lacking. For example, when considering SourceLocations, some functions in the Sema randomly don’t accept invalid ones, which hinders their use when invoking them externally, and I had … fun … trying to create my own SourceLocations that point to external locations.

The existing CodeGen modules were never designed, as far as I’m aware, to interface with external code generation, but in my experience, they can be made to do so relatively easily in most cases.

Currently, I’ve been operating on Itanium systems only (Ubuntu x64 and Windows x86 with MinGW) but I figure that it’s a good start since Clang’s own support for MSVC ABI is limited. I don’t have any other targets available to test for. As for tests themselves, as far as I’m aware Clang uses a totally different testing strategy to me, so whilst I have a couple of basic existing tests, I’m not sure how they can be adapted.

Officially, IR-level interoperation with Clang isn’t supported at all right now, and the CodeGen interfaces are private. I figure that before I start working on patches and providing interfaces, I want to find out if this is something that the Clang devs in general want to move forward with. My machine is very slow and building LLVM/Clang takes some time, so I’d really rather not get cracking only to find out that my patches are unwanted.

I would love to see this as an official part of clang with a stable[1] (C or C++, don't care) public interface for external consumers. Sign me up to review patches!

David

[1] Or, at least, mostly stable. i.e. backwards source compatible between releases, with one full release of deprecation before old APIs are removed.

Currently, I have maintained the same interface from 3.2 to 3.4, and it’s fairly abstract so I’m feeling fairly confident that I won’t need interface changes in 3.5, but I won’t update until stable is done.

Usually when people talk about wanting an “ABI library” for LLVM, they are writing their own language frontend and merely want logic for C ABI lowering so they can use libraries with C interfaces. They probably don’t want to link in Clang’s entire AST and CodeGen libraries, for example. They want something lightweight that only understands C types: structs, unions, complex, floating point types, vectors, arrays, and integers. I think Julia wants something like this, for example.

What you’re talking about doesn’t sound like an ABI library so much as improving Clang’s library interfaces to the point that it can be used as a library to emit IR that interoperates with C++. This sounds like a good general goal to me. You could use it to build something superior to SWIG, for example.

Yes, that's right. We (Julia) have an implementation partly based on
the one used by LDC, but it would great to have a library that allows
you to give to callbacks to classify your types according to the
distinction that the C ABI makes (the rest hopefully being possible to
infer from the llvm struct types themselves).

We also have a package that pulls in Clang and uses it's code
generation to interoperate with C++, so we'd love to see those
interfaces made public, since we'll then be able to use a shared
library rather than having to link our own with those interfaces made
forcibly public.