[Progress Update] LLVM Runtimes Subdirectory

Project Infrastructure LLVM Dev List Archives
1

Hi LLVM-Dev,

Over the past week or so I’ve been working on build system improvements around the LLVM runtimes directory. You may remember this from the LLVM-Dev discussion ([llvm-dev] [RFC] LLVM Directory Structure Changes (was Re: [PATCH] D20992: [CMake] Add LLVM runtimes directory)), or the patch review discussion (⚙ D20992 [CMake] Add LLVM runtimes directory) back in June.

Recently, I’ve made a handful of new changes that actually make it useful, and I wanted to blast out this update and encourage people to give it a try.

Let me start with an update on the current state.

First and foremost, this is not currently supported with multi-configuration generators (Visual Studio or Xcode). I’ve done my testing with Ninja, but I expect it will work the same with Make.

Currently the runtimes directory is supported for building compiler-rt and libcxx. You can enable the new build paths either by putting your compiler-rt and libcxx checkouts into the runtimes subdirectory or by symlinking the directories. I do plan to support variables for overriding paths to the runtimes similar to how we support the LLVM projects directory today, but I haven’t gotten there yet (let me know if this prevents you from using the feature and I’ll prioritize it).

With support for libcxx I made the runtimes build generate top-level targets for each runtime project. The runtime project targets are named for the project with leading ‘lib’ prefixes removed (i.e. compiler-rt, cxx). Also note, for the runtimes subdirectory compiler-rt is treated as a separate project from the builtins. One of the current limitations of this system is that I still don’t have a solution for building and running the builtins tests.

I’ve also added support to the runtimes for generating a target that runs all runtime tests from a single lit invocation. This target is exposed in the build as “check-runtimes”, and it is connected as a dependency of LLVM’s “check-all” target. This makes it so “check-all” includes the runtime test, but not under the same lit invocation. I have ideas for how to fix that which I’ll be working on.

The last new feature I’ve added is the ability to expose a set of targets from runtime projects into the top-level generated build files. I’ve connected this new feature into compiler-rt, which allows exposing targets like asan and check-asan through to the top-level build. This last feature has some slight awkwardness and I’m curious to hear opinions on it.

To avoid having to hard-code a list of targets to expose I made the runtime configuration step generate a list of targets to map back. This means that before you can use targets from the runtimes build you need to build enough of the toolchain to configure the runtime project build. What that means is you can’t do the following in a new build tree:

cmake ...
ninja asan

Instead you would do:

cmake ...
ninja runtimes-configure
ninja asan

I can’t think of any good ways around this. I’m curious if anyone has ideas, or opinions on whether or not this is an acceptable workflow.

The big items on my to-do list are:
* Running the builtins tests
* Porting libcxxabi and libunwind
* Getting check-all with a single lit invocation
* IDE support
* Supporting specifying the path of runtime libraries

In general I think this is getting to a point where people can give it a test drive and see what they think, and I’m eagerly looking for feedback.

Thanks,
-Chris

2

Recently, I’ve made a handful of new changes that actually make it useful, and I wanted to blast out this update and encourage people to give it a try.

Woot! Compiling RT as we speak... :slight_smile:

Interesting, "ninja runtimes-configure" wants to build the whole
thing... Is that because you use the just-built clang to build the
multiple targets for each runtime?

What's the CMake variable to enable RT for ARM, even though I'm
running on x86_64?

cmake ...
ninja runtimes-configure
ninja asan

It does seem awkward. I don't know CMake well enough to know what's
the problem, much less how to fix it, sorry. :frowning:

In pure Make, I'd just make the asan rule depend on the other, maybe
the ninja file could be changed in that way?

The big items on my to-do list are:
* Running the builtins tests
* Porting libcxxabi and libunwind
* Getting check-all with a single lit invocation
* IDE support
* Supporting specifying the path of runtime libraries

Sounds like a solid plan.

cheers,
--renato

3

Right, all went well until I got this:

[0/1] Re-running CMake...
-- Builtin supported architectures: i386;x86_64
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/rengolin/devel/llvm/workdir/libs/build/runtimes/builtins-bins
ninja: no work to do.
[677/681] No install step for 'builtins'
[680/681] Performing configure step for 'runtimes'
-- Compiler-RT supported architectures: x86_64;i386
CMake Error at CMakeLists.txt:93 (list):
  list sub-command REMOVE_DUPLICATES requires list to be present.

CMake logs attached.

--renato

CMakeLogs.zip (3.41 KB)

4

Recently, I’ve made a handful of new changes that actually make it useful, and I wanted to blast out this update and encourage people to give it a try.

Woot! Compiling RT as we speak... :slight_smile:

Interesting, "ninja runtimes-configure" wants to build the whole
thing... Is that because you use the just-built clang to build the
multiple targets for each runtime?

Yes, because it uses the just-built tools, clang is a dependency of the configure step.

What's the CMake variable to enable RT for ARM, even though I'm
running on x86_64?

This isn’t explicitly supported yet, but the standard COMPILER_RT_* prefixed options will be passed through.

cmake ...
ninja runtimes-configure
ninja asan

It does seem awkward. I don't know CMake well enough to know what's
the problem, much less how to fix it, sorry. :frowning:

In pure Make, I'd just make the asan rule depend on the other, maybe
the ninja file could be changed in that way?

The problem is that the top-level CMake doesn’t actually know what targets to generate until after you configure compiler-rt because the capabilities of the compiler you build will impact what targets get generated.

The big items on my to-do list are:
* Running the builtins tests
* Porting libcxxabi and libunwind
* Getting check-all with a single lit invocation
* IDE support
* Supporting specifying the path of runtime libraries

Sounds like a solid plan.

-Chris

5

Woot! Compiling RT as we speak... :slight_smile:

Right, all went well until I got this:

[0/1] Re-running CMake...
-- Builtin supported architectures: i386;x86_64
-- Configuring done
-- Generating done
-- Build files have been written to:
/home/rengolin/devel/llvm/workdir/libs/build/runtimes/builtins-bins
ninja: no work to do.
[677/681] No install step for 'builtins'
[680/681] Performing configure step for 'runtimes'
-- Compiler-RT supported architectures: x86_64;i386
CMake Error at CMakeLists.txt:93 (list):

I should add a catch for this. It might be caused by a slightly out of date compiler-rt. Do you have r279863?

-Chris

6

I put in a fix that should prevent that configuration error in r279893.

-Chris

7

Hi Chris,

Your fix made it work, I managed to build all of it to the end. Thanks!

Yes, because it uses the just-built tools, clang is a dependency of the configure step.

Perfect.

This isn’t explicitly supported yet, but the standard COMPILER_RT_* prefixed options will be passed through.

Right, I'm playing with the flags and not getting anywhere. I tried:

-DCOMPILER_RT_DEFAULT_TARGET_ARCH=
-DCOMPILER_RT_SUPPORTED_ARCH=
-DCOMPILER_RT_BUILTINS_STANDALONE_BUILD=
-DBUILTIN_SUPPORTED_ARCH=

I noticed that RT's builtin-config-ix.cmake only has cross options for
Darwin, would that be the reason?

The problem is that the top-level CMake doesn’t actually know what targets to generate until after you configure compiler-rt because the capabilities of the compiler you build will impact what targets get generated.

Hum, that sounds like a chicken and egg problem. Maybe some additional
CMake flag to facilitate the cross options of RT's builtin (as we
discussed above) could "assume" the targets directly.

A first implementation would fail later if Clang can't target the arch
you asked (ex. if you removed it from LLVM_TARGETS_TO_BUILD), but an
intersection check on both lists could yield some pre-checks to make
it a cmake error instead.

cheers,
--renato

8

Hi Chris,

Your fix made it work, I managed to build all of it to the end. Thanks!

Yes, because it uses the just-built tools, clang is a dependency of the configure step.

Perfect.

This isn’t explicitly supported yet, but the standard COMPILER_RT_* prefixed options will be passed through.

Right, I'm playing with the flags and not getting anywhere. I tried:

-DCOMPILER_RT_DEFAULT_TARGET_ARCH=
-DCOMPILER_RT_SUPPORTED_ARCH=
-DCOMPILER_RT_BUILTINS_STANDALONE_BUILD=
-DBUILTIN_SUPPORTED_ARCH=

I noticed that RT's builtin-config-ix.cmake only has cross options for
Darwin, would that be the reason?

On non-Darwin it should call into `test_targets()` which should be running compile tests. This build logic is all new this year, and may not be as well tested on non-Darwin platforms. I’ll see if I can poke at it a bit and see if I can figure out the magic incantations and maybe clean it up a bit.

The problem is that the top-level CMake doesn’t actually know what targets to generate until after you configure compiler-rt because the capabilities of the compiler you build will impact what targets get generated.

Hum, that sounds like a chicken and egg problem. Maybe some additional
CMake flag to facilitate the cross options of RT's builtin (as we
discussed above) could "assume" the targets directly.

I think this would work well for the builtins, but less so for the sanitizers. The logic for determining when to build the sanitizers is sadly non-trivial. It might be possible to refactor compiler-rt’s sanitizer enabling logic so that we could execute it directly from the LLVM configuration to get out a list of targets. I may look into that if I have some time.

A first implementation would fail later if Clang can't target the arch
you asked (ex. if you removed it from LLVM_TARGETS_TO_BUILD), but an
intersection check on both lists could yield some pre-checks to make
it a cmake error instead.

Handling the case of architecture support for the builtins is probably the easiest case that we could make some educated decisions around. I honestly haven’t even thought too far into that because I haven’t tried adding support for multiple builtin architectures yet. That said, when we get there it is a great idea to validate the architectures against LLVM_TARGETS_TO_BUILD to ensure that you at least stands a chance of getting a working configuration.

-Chris