llvm-project has traditionally supported the so-called “standalone builds”. I cannot find good documentation but it appears to be something like the following, specifying a toplevel project directory as CMake’s -S argument. Unfortunately with current main branch I get some errors:
% cmake -Sclang -B/tmp/out/clang -GNinja
...
CMake Error at CMakeLists.txt:89 (include):
include could not find requested file:
GetErrcMessages
...
CMake Error at cmake/modules/AddLLD.cmake:17 (get_target_export_arg):
Unknown CMake command "get_target_export_arg".
Call Stack (most recent call first):
Common/CMakeLists.txt:29 (add_lld_library)
Using lld/ as the source directory breaks similarly:
% cmake -GNinja -Slld -B/tmp/out/lld
CMake Error at CMakeLists.txt:89 (include):
include could not find requested file:
GetErrcMessages
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB
...
CMake Error at cmake/modules/AddLLD.cmake:17 (get_target_export_arg):
Unknown CMake command "get_target_export_arg".
Call Stack (most recent call first):
Common/CMakeLists.txt:29 (add_lld_library)
If nobody stands up to support the standalone build, or if the involved efforts are too high, perhaps we should official deprecate the standalone build for clang and lld as well. (For lld) it seems to me that the standalone build has multiple issues for a couple of releases now.
What does a standalone build offer that a regular LLVM_ENABLE_PROJECTS build cannot do? This needs to consider the deprecation/removal of libc++/libc++abi/libunwind standalone builds since clang may probably be built multiple times with a standalone build.
Can the regular LLVM_ENABLE_PROJECTS build be enhanced to meet the requirement?
Thank you for starting this thread. The first thing I’ll note is that the standalone builds require having an installed version of LLVM somewhere on your system that CMake can find. I’m not sure if you have that on the system you are testing on, but if you don’t that’s why the builds are failing.
We use standalone builds when packaging the LLVM sub-projects for Fedora.
I don’t think so, because this requires having the llvm source directory be present on the system, and the whole purpose of standalone builds is to use the installed version of llvm and not one from the source tree.
I know that the runtimes are moving away from this kind of configuration, but I think it’s still useful for clang, lld, and other projects that depend on the LLVM libraries. I do think, though, if we want to keep stand-alone builds as a supported configuration, we need better documentation for it and also a buildboot.
The downside of supporting standalone build is the increased complexity of the build system. So I guess the question is, ignoring the runtime builds which have already switched, is supporting the standalone builds causing problems for people trying to work on the build systems for clang, lld, lldb, etc?
Standalone builds are crucial to Gentoo. Gentoo is a highly configurable source distribution, and I can’t even imagine building LLVM from a single huge blob.
Non-standalone build would be:
Confusing. Right now options specific to every project are kept in the project itself. With the “huge blob” approach, we’d have to have a single ebuild with tons of options that apply only to some subprojects, and users would end up switching options that don’t do anything because they didn’t enable the project in question.
Detrimental to dependencies. Right now, packages depend only on projects they actually need. If you need only LLVM, you build LLVM. If you need Clang and have LLVM installed already, you build Clang. So on and so on. With the “huge blob” approach, people would either have to always build the tools they don’t actually need, or spend significant time rebuilding everything to install an additional tool they need.
Detrimental to testing. Right now, it’s pretty common for test suites of openmp or sanitizers to fail, and with package building it’s not something you resume in the end. We’re basically talking that if a test suite at the very end of testing process fails, we have to start over.
I don’t get the claim that standalone builds are not support by anyone. I’ve been submitting patches to fix any regressions (and they weren’t big patches) for years now. If anything, I’m seeing a trend towards improved support. A few years back, I was entirely alone doing this — today I see other people testing and fixing it, I see people quickly fixing regressions I report.
I wish I could do more to help but I simply don’t have the time or the resources to set up a buildbot for this. Nevertheless, I’m spending significant effort testing new releases and the main branch as often as I can.
Removing the support for standalone builds would be a huge blow to Gentoo, and I don’t think I’ll be able to continue maintaining LLVM if that happens. At least not in the current form, and seeing how many packages actually depend on different LLVM projects, I don’t think any other form is feasible.
I don’t require support for standalone builds myself (unless “debugging” CMake scripts), but quite a few Flang core developers use them for their day-to-day work.
The build steps are documented here. There’s also a buildbot: flang-aarch64-out-of-tree (“out-of-tree” should probably be replaced with “standalone” at some point).
I’ve been maintaining the LLDB standalone build. The main motivation for us is Swift, which builds llvm, clang, swift and lldb as separate projects through a wrapper script called build-script.
I’m a bit light on the details, but IIRC the runtimes were always built differently with build-script so removing support for building them standalone wasn’t an issue, but that won’t be true for the “regular” projects.
PS: Slightly less important, but for people wanting to use the Xcode CMake generator, the standalone builds are essentially the only way to end up with a workable workspace. I think this is mostly a quality-of-implementation issue for the CMake generator itself and it had something to do with it generating a huge amount of target. Maybe things have gotten better over the last few years (I don’t use it myself) but last I checked generating an “monolithic” Xcode project was basically unusable.
If standalone builds were maintained, tested in CI, and didn’t change without warning I’d consider switching back to them for FreeBSD ports. I gave up on them years ago because from my perspective they kept breaking or changing in erratic ways and have since been building one big blob with many optional knobs while waiting for someone to add pkg features to allow the individual parts to be split out. Switching would be a major undertaking and cause user confusion though so there would need to be a strong commitment going forward.
As a packager, another benefit of standalone builds is that if I want to backport a patch on clang, I don’t need to rebuild LLVM, so it saves time and energy. And things are obviously worse for flang which has a deeper dependency tree.
I’m using the standalone build for LLVM snapshots for Fedora. I’ve recently landed ⚙ D118481 Add cmake to source release tarballs in order to address the problem of a shared cmake directory on the top-level directory level. This directory didn’t exists in any standalone source tarball before but now it does.
I’m monitoring the snapshots on a regular basis and up until last week everything worked pretty fine with upstream. I’m in the middle of rebasing some downstream patches and them I’m going to see if I run into similar problems.
For me having the standalone builds drastically reduces the turnaround times for delivering packages when something is rotten. Suppose compiler-rt was broken and I have to fix something in the package blueprints in order to get it back to work. Then I have to build llvm and clang at least and that costs too much time. Multiply this by the number of platforms (3always) and architectures (5) that I support and then I’ll have a valid argument to provide packages fast without jeopardizing my other real work.
Apart from the above I think it adds clarity on the packaging level to have standalone builds available.
I’m willing to investigate if I can come up with a buildbot configuration that tests standalone builds for llvm and clang at first. That configuration will probably be closely look like the cmake config we use when we build for packages for Fedora.