Compiler-RT roadmap

Hello,
In continuation of several threads from the last week, I’d like to ask: is there a stated plan of what is going to happen with compiler-rt in the near future? In particular, I’m interested if any of the following is planned to happen:

  • Separation from clang
    I’ve seen a suggestion to rename compiler-rt to “libclang_rt”, but its’ applicability is much broader than just clang. I think it would make more sense to make it more independent of clang, not less. If anything, it should be renamed “llvm-rt”, because it’s LLVM codegen that emits references to functions defined in compiler-rt.
    If I understand compiler-rt’s build system correctly, right now the only supported way of building it is as a part of clang build (however it must be checked out into llvm/projects directory?) If it is to be used with other LLVM-based compilers, it needs to be build-able separately from clang.

  • Being able to build it for all platforms that LLVM can target

Since LLVM-produced binaries depend on compiler-rt, it should be available for all LLVM target platforms. This seems not to be the case currently (at least via Makefiles, maybe it’s possible via cmake, but I have not been able to make it work on Mingw/Windows).

  • Inclusion of libunwind

It was also suggested that libunwind should be moved from libcxxabi into compiler-rt, because it isn’t C++ specific. To me, it seems like the correct decision, because LLVM generates direct calls to _Unwind_Resume for any code that uses ‘invoke’ instructions and cleanup landing pads.
Is anyone already working on this? And if not, do compiler-rt maintainers agree that this is the right thing to do, and will they accept patches?

I’ve developed some patches that try to address #1 and #2 above: http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20140203/203928.html Can somebody please take a look? Who are the current maintainers of compiler-rt?

thanks,
Vadim

- Separation from clang
I've seen a suggestion to rename compiler-rt to "libclang_rt", but its'
applicability is much broader than just clang. I think it would make more
sense to make it more independent of clang, not less. If anything, it
should be renamed "llvm-rt", because it's LLVM codegen that emits references
to functions defined in compiler-rt.

I'm not directly involved in the compiler-rt project, but I think this
is a very good point.

- Being able to build it for all platforms that LLVM can target
Since LLVM-produced binaries depend on compiler-rt, it should be available
for all LLVM target platforms. This seems not to be the case currently (at
least via Makefiles, maybe it's possible via cmake, but I have not been able
to make it work on Mingw/Windows).

This is, unfortunately, a manual process. The library is being
developed mainly for x86_64, and not much more. I'm making them build
for ARM, but I'm still not sure it works. :wink:

I'll update the status later, when I get Clang to recognize --rtlib on
non-Darwin systems.

- Inclusion of libunwind
It was also suggested that libunwind should be moved from libcxxabi into
compiler-rt, because it isn't C++ specific. To me, it seems like the
correct decision, because LLVM generates direct calls to _Unwind_Resume for
any code that uses 'invoke' instructions and cleanup landing pads.
Is anyone already working on this? And if not, do compiler-rt maintainers
agree that this is the right thing to do, and will they accept patches?

I think that was the rationale, yes, and one that I and many people agree.

I've developed some patches that try to address #1 and #2 above:
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20140203/203928.html
Can somebody please take a look? Who are the current maintainers of
compiler-rt?

Unfortunately, I can't review them, especially for Windows. Too far

cheers,
--renato

- Separation from clang
I've seen a suggestion to rename compiler-rt to "libclang_rt", but its'
applicability is much broader than just clang. I think it would make more
sense to make it more independent of clang, not less. If anything, it
should be renamed "llvm-rt", because it's LLVM codegen that emits references
to functions defined in compiler-rt.

I'm not directly involved in the compiler-rt project, but I think this
is a very good point.

If the fragments contained in the rt are genuinely disconnected from clang's codegen, this seems like a great idea, and would make the disconnection clearer by the renaming.

- Being able to build it for all platforms that LLVM can target
Since LLVM-produced binaries depend on compiler-rt, it should be available
for all LLVM target platforms. This seems not to be the case currently (at
least via Makefiles, maybe it's possible via cmake, but I have not been able
to make it work on Mingw/Windows).

This is, unfortunately, a manual process. The library is being
developed mainly for x86_64, and not much more. I'm making them build
for ARM, but I'm still not sure it works. :wink:

I'll update the status later, when I get Clang to recognize --rtlib on
non-Darwin systems.

we are also thinking about how to achieve this (both at build-time and to query supported targets at runtime) ...
... but first it seems essential to try and get some consensus on toolchain layout so that different people's work in this area don't conflict. We will be suggesting a BOF at Edinburgh to provide an opportunity for folks to try and thrash this out (with a starting proposal)

- Inclusion of libunwind
It was also suggested that libunwind should be moved from libcxxabi into
compiler-rt, because it isn't C++ specific. To me, it seems like the
correct decision, because LLVM generates direct calls to _Unwind_Resume for
any code that uses 'invoke' instructions and cleanup landing pads.
Is anyone already working on this? And if not, do compiler-rt maintainers
agree that this is the right thing to do, and will they accept patches?

I think that was the rationale, yes, and one that I and many people agree.

As part of the compiler-rt tree is a better home than libcxxabi also completely agree.

However, I still think it's necessary to make inclusion of the unwinder (in a given build) optional, or to allow it to be built as a separate module, since on many systems clang/llvm is an add-on compiler where we might want to use the code-fragments in "llvm-rt" in conjunction with an external unwinder which is part of an existing system.

I've developed some patches that try to address #1 and #2 above:
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20140203/203928.html
Can somebody please take a look? Who are the current maintainers of
compiler-rt?

not sure about specifically for compiler-rt
for config & make - you might ask Eric Christopher (echristo)
for cmake you might try Takumi Nakamura (chapuni)

if they can't help, perhaps they can suggest the right person.

Unfortunately, I can't review them, especially for Windows. Too far
from my area of expertise. :wink:

likewise, unfortunately.

On these systems, I'd imagine that we'd also most likely use the implementations of a number of builtins from whatever the system provided.

I don't really see the compiler-rt code as something that I'd expect people to build as-is. Both FreeBSD and Apple pull code from there, but neither of us use the upstream build system. In OS X I think they mostly go into libSystem.dylib, in FreeBSD they go into libgcc and friends (these are increasingly llvm code and not gcc code, but the names are retained for historical reasons. Eventually we'll probably turn /lib/libgcc_s.so.1 into a symlink to something else).

I guess there are three use cases for the compiler-rt code:

- You're producing a completely stand-alone development environment for an OS (e.g. something like MinGW) and you need to have all of the support functions, so you build compiler-rt and you link it into your programs.

- You're building on a system that provides support libraries but doesn't implement everything that your LLVM-based compiler needs, so you take parts of compiler-rt to fill in the gaps.

- You're building the core libraries for an OS, so you take the code and put it wherever makes the most sense for your architecture.

I think that having the unwind code as a separate library under compiler-rt helps all of these cases, so I'm definitely not opposed to that. I wonder if we need to go further and make more symbols weak so that they can be replaced more easily.

David

>> - Separation from clang
>> I've seen a suggestion to rename compiler-rt to "libclang_rt", but its'
>> applicability is much broader than just clang. I think it would make
more
>> sense to make it more independent of clang, not less. If anything, it
>> should be renamed "llvm-rt", because it's LLVM codegen that emits
references
>> to functions defined in compiler-rt.

Why? After all, the current project name (compiler-rt) doesn't tie the code
to Clang and shows
that the libraries contained there are generally not useful in a standalone
version, and the calls into
these libraries are expected to be emitted by (some) compiler.

>
> I'm not directly involved in the compiler-rt project, but I think this
> is a very good point.

If the fragments contained in the rt are genuinely disconnected from
clang's codegen, this seems like a great idea, and would make the
disconnection clearer by the renaming.

- Being able to build it for all platforms that LLVM can target

>> Since LLVM-produced binaries depend on compiler-rt, it should be
available
>> for all LLVM target platforms. This seems not to be the case currently
(at
>> least via Makefiles, maybe it's possible via cmake, but I have not been
able
>> to make it work on Mingw/Windows).
>
> This is, unfortunately, a manual process. The library is being
> developed mainly for x86_64, and not much more. I'm making them build
> for ARM, but I'm still not sure it works. :wink:
>
> I'll update the status later, when I get Clang to recognize --rtlib on
> non-Darwin systems.

we are also thinking about how to achieve this (both at build-time and to
query supported targets at runtime) ...
... but first it seems essential to try and get some consensus on
toolchain layout so that different people's work in this area don't
conflict. We will be suggesting a BOF at Edinburgh to provide an
opportunity for folks to try and thrash this out (with a starting proposal)

>> - Inclusion of libunwind
>> It was also suggested that libunwind should be moved from libcxxabi into
>> compiler-rt, because it isn't C++ specific. To me, it seems like the
>> correct decision, because LLVM generates direct calls to _Unwind_Resume
for
>> any code that uses 'invoke' instructions and cleanup landing pads.
>> Is anyone already working on this? And if not, do compiler-rt
maintainers
>> agree that this is the right thing to do, and will they accept patches?
>
> I think that was the rationale, yes, and one that I and many people
agree.

As part of the compiler-rt tree is a better home than libcxxabi also
completely agree.

However, I still think it's necessary to make inclusion of the unwinder
(in a given build) optional, or to allow it to be built as a separate
module, since on many systems clang/llvm is an add-on compiler where we
might want to use the code-fragments in "llvm-rt" in conjunction with an
external unwinder which is part of an existing system.

>> I've developed some patches that try to address #1 and #2 above:
>>
http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20140203/203928.html
>> Can somebody please take a look? Who are the current maintainers of
>> compiler-rt?

not sure about specifically for compiler-rt

I don't know who currently maintains the set of builtins (defined in
/compiler-rt/lib/*.c) files.
I can review the CMake parts and linux-specific Makefile parts.

Hello,
In continuation of several threads from the last week, I'd like to ask: is
there a stated plan of what is going to happen with compiler-rt in the near
future? In particular, I'm interested if any of the following is planned
to happen:

- Separation from clang
I've seen a suggestion to rename compiler-rt to "libclang_rt", but its'
applicability is much broader than just clang. I think it would make more
sense to make it more independent of clang, not less. If anything, it
should be renamed "llvm-rt", because it's LLVM codegen that emits
references to functions defined in compiler-rt.

The libraries produced are already called libclang_rt.<arch>.so and have
been shipped, and I think it's too late to change that name.

If I understand compiler-rt's build system correctly, right now the only
supported way of building it is as a part of clang build (however it must
be checked out into llvm/projects directory?) If it is to be used with
other LLVM-based compilers, it needs to be build-able separately from clang.

Making it build separately seems reasonable.

- Being able to build it for all platforms that LLVM can target
Since LLVM-produced binaries depend on compiler-rt, it should be available
for all LLVM target platforms. This seems not to be the case currently (at
least via Makefiles, maybe it's possible via cmake, but I have not been
able to make it work on Mingw/Windows).

You can build the asan runtime for Windows with CMake, but I think that's
it.