Using LLVM ADT/Support lib in external projects without pulling full LLVM monorepo

[Hardcode84]

Hello. I find LLVM Support/ADT library quite nice to use and I was experimenting recently to extract it to standalone project, to be usable without pulling entire LLVM monorepo as a dependency.

Repo: GitHub - Hardcode84/llvm-adt

Implementation details:

• There is a script llvm-adt/scripts/update.py at master · Hardcode84/llvm-adt · GitHub to pull relevant LLVM sources (headers, cpps and tests) to the repo.
• There are googletest/googlemock sources pulled from LLVM repo for testing llvm-adt/third-party/unittest at master · Hardcode84/llvm-adt · GitHub (not updated via script)
• Initially I only wanted STLExtras and SmallVector but SmallVector and tests pulled a lot of lib dependencies and I decided to pull the entire library.
• There is a GH action to run the update script and tests every night with current LLVM HEAD llvm-adt/.github/workflows/update-llvm-sha.yml at master · Hardcode84/llvm-adt · GitHub.
• LLVM proper has the llvm-config.h which is generated by CMake, for this project I just hardcoded it for linux, for now llvm-adt/include/llvm/Config/llvm-config.h at master · Hardcode84/llvm-adt · GitHub
• Some tests are depending on LLVM IR, unfortunately, and skipped, but the Support lib itself is mostly looks fine in this regard (but we have cases when C++ API depends on C API).

Also, I was experimenting with amalgamating all Support sources in single cpp, mostly to have a single file to add into the project and not bother with the cmake.
Surprisingly, this mostly worked out of the box and required only trivial fixes to upstream (which are mostly beneficial by itself):

• [support] SlowDynamicAPInt: use fully qualified namespaces for func definitions by Hardcode84 · Pull Request #132575 · llvm/llvm-project · GitHub
• [support][nfc] MD5: Undef macros after use by Hardcode84 · Pull Request #132132 · llvm/llvm-project · GitHub
• Few files are still skipped llvm-adt/scripts/update.py at master · Hardcode84/llvm-adt · GitHub, mostly because they have same-named functions/static vars in different modules.

I was told there is a general desire to pull Support lib from LLVM subtree to top-level project within monorepo, and something of this may be useful.

[mshockwave]

I always want something like this, thanks!
On the technical side: I thought LLVMSupport still depends on LLVMDemangle? Did you also pull that in?

[Hardcode84]

Yeah, Demangle is skipped for now

[rnk]

This might be interesting, but I don’t think there’s necessarily community consensus to do this.

Perhaps a first step in this direction would be to break LLVM Support out of the llvm subproject, and move it to it’s own proper top-level directory subproject. I don’t know if this remains true today, but MLIR people used to tell me “MLIR doesn’t really depend on LLVM, it really only uses Support”. Historically, each top-level subproject has maintained some support for building in “standalone mode”, which is ultimately what powers the new “runtimes builds” for libc++, compiler-rt, and the others. What you’re proposing is very much like an LLVMSupport standalone build.

I’ve also been thinking about trying to promote the LLVM and Clang dylib builds as the default build configuration, because the statically linked build configuration is so resource hungry, and most distros ship the dylib build (although I haven’t confirmed this). You might notice that in the dylib build configuration we have to do funny things to link LLVM Support statically or as a separate shared library for llvm-tblgen or to handle other edge cases.

All of that ladders up to about 50% of a case to make Support it’s own subproject, but IMO it doesn’t push it over the top. I’m not planning to work on this and I doubt there is consensus to do it, but if anyone ever did want to walk this path, this is the way I’d recommend.