[Proposal] split built-ins from the rest of "compiler-rt"

[rengolin]

Quick summary

Trying to summarise the feedback, there seem to be two major trends in this thread:

1. There’s good support for splitting builtins from the rest of compiler-rt. Supporting arguments were mostly in the build side, and range from simplifying the build system, decoupling it from the requirements of the more complex runtimes in RT, inter-dependencies creating bootstrapping problems, etc.
2. There’s a split between carrying runtime library information across the toolchain and linking (at least parts) of the builtins by default, with technical arguments on both sides.

These concerns could be seen as orthogonal, so I think we should (at least try to) treat them separately.

My view is that how we split this will have to take more than just individual pros and cons technical arguments and venture into usability and “least surprise” principles. It also can’t be based on “what GCC does” (or doesn’t do). LLVM is not a compiler, but a lot of compilers are built on top of it, so we need to at least provide the ability of toolchain builders (including upstream Clang/LLVM) to provide users with reasonable functionality.

Split builtins

I think there’s enough support to do this, mostly due to build issues, and there should be little to no issues to the rest of the compiler, including the issue that created this proposal. Splitting it won’t force us to include it by default on any target, it’ll just simplify the build system.

@compnerd @petrhosek (and others interested), we should discuss next steps towards an actual RFC on the final structure.

The key issues in this thread need to be addressed, as well as the potential of linking with parts of the builtins (ex. 128-bit arithmetic) but not the whole, for specific targets.

Note: The runtime library’s name should still continue to be compiler-rt (because of --rtlib) and so we may keep the builtins in compiler-rt and move the rest somewhere else, which is the same thing.

Link builtins by default or not (or parts)

This is less clear, and I see three options:

1. Always link builtins and default --rtlib=compiler-rt on targets (CPU+OS+ABI) that we support all Clang generated functions. Other targets just do what we “know” the target runtime supports.
2. Split builtins into “generic support for the targets” and “Clang-specific things” and always link the latter, keeping the former in the same way we do today with --rtlib.
3. Try to propagate options through driver, front-end, IR, MIR, objects, linker, libraries, etc. This is a non-trivial amount of work, involving a non-trivial number of RFCs on almost all stages of the compilation process, and will probably need to be agreed in full before anything starts to happen.

As is probably clear by my wording above, I’m ok with the first two, but I think the third option is a recipe for never finishing and being blocked by future changes to the GNU toolchain (who usually ignore our decisions and do their stuff regardless).

If my count is right, in this thread (which is no proof of consensus), there’s more support for 1 and 2 than 3. But of course, this will depend on how we implement them in the long run.

Note: if we make compiler-rt the default runtime for specific platforms, we have to emit build warnings when the compiler-rt project is not chosen. And we’ll start getting issues open on that…

Discarded ideas

There are, of course, many more ideas, but they were discarded due to not being practical.

For example, relying on incomplete information to drive the compiler. Taking a driver triple to assume support, but not knowing what the linker will do (because we forgot to add -Wl options).

Or assuming we’re linking with the compiler driver and pass the triple to the linker flags, only the linker ends up being bfd who has no idea what that is and will ignore.

Also linking the builtins as well as the system’s runtime (ex. libgcc) seems frail. Sure, there are options on symbols (like COMDAT) that help making the decision of which one to take, and there’s ordering of library that we can control.

But there’s always tools that don’t quite support what we need, and argument ordering can always be broken by a system-wide CXXFLAGS or users adding stuff to the end of the command line on a script.

Next steps

I’d like to propose some next steps, so that we’re clear on where we stand.

Builtins split

There were enough details on this thread to form a good technical proposal, so we should make a concrete RFC and start discussing actual technical details.

This doesn’t mean consensus has formed, just that there’s enough support to spend time on an actual RFC. The decision to continue with the split should be made there, not here.

Linking rt-lib

Before we decide how we’ll link what we need when, we need some information from people using compiler-rt in different ways.

Already using compiler-rt as default builtin library

For folks already using it (FreeBSD, Mandriva, Apple), what would simplify your build system?

I imagine there are hacks to pretend to be libgcc in places, but it would be nice if one of those places isn’t LLVM. If we split the builtins, what would be an easy way to make that the default for your platform without the need of hacks?

Able to use compiler-rt but problems with system runtime

For Win/GNU-based environments, libgcc is still there. Are there any tools, code size, special requirements that stop you linking compiler-rt by default with Clang on your system?

Think about compiling objects with different compilers (Clang and GCC) and linking them together. Also, linking GCC compiled static/dynamic system libraries with Clang compiled user programs. Weird system requirements around import/export symbols, system linker oddities, etc.

If not, could you do a comprehensive build with --rtlib=compiler-rt today and see if all works? If it does, I guess we can “just turn it on by default” on the affected triples and we’d have less problems.

If so, would linking just the shim delta (ex. 128-bit) as a separate library name be a reasonable alternative solution?

Unable to use compiler-rt due to support quality

If your target (mostly new architecture I guess) doesn’t have full support in compiler-tr (both C and asm functions) for the functionality your system needs, we should not be generating calls to compiler-rt only functions in the first place.

The issue reported was on AArch64, but that’s a back-end decision, and your target shouldn’t make that call legal, and you’ll have no problems.

But as soon as you want to use those functions, you’ll be in the same place and will have to make a decision.

Anything else I forgot…

Please include your case here.

Thanks!

[mstorsjo]

@rengolin

There’s good support for splitting builtins from the rest of compiler-rt. Supporting arguments were mostly in the build side, and range from simplifying the build system, decoupling it from the requirements of the more complex runtimes in RT, inter-dependencies creating bootstrapping problems, etc.

Note, that from the build perspective, it is already somewhat split. You can build compiler-rt builtins only, by pointing cmake at compiler-rt/lib/builtins. The runtimes build does this, and so does other builds of toolchains that bootstrap from scratch, where the builtins need to be built before anything else, so that the cmake setup for everything else will work as expected.

I’m not sure what more there is to gain. From a clarity point of view, things aren’t necessarily ideal, but I don’t see how changing it would be worth the churn for everybody who currently are building it. (If they’re moved out, it’d break the build scripts for anyone assembling a toolchain, expecting to be able to build it like it is now. Thus, moving it probably requires some temporary glue to let the old way of building it still work for a transition period.)

@rengolin

Try to propagate options through driver, front-end, IR, MIR, objects, linker, libraries, etc. This is a non-trivial amount of work, involving a non-trivial number of RFCs on almost all stages of the compilation process, and will probably need to be agreed in full before anything starts to happen.

I’m not sure who suggested this, but this sounds like a massively more complex thing than what I and others were suggesting. The other collective suggestion wouldn’t need coordination with GNU toolchains or anything else - it would be another codegen flag that gets added to the LLVM codegen phase, which affects what code is generated for those operations (libcalls to builtins or less efficient inline code). This still probably isn’t trivial and some didn’t quite like it (it needs to add a per-module flag and tracked properly for e.g. LTO), but I feel this point is discarding the suggestion entirely by making it one magnitude harder than it should be.

@rengolin

Already using compiler-rt as default builtin library

For folks already using it (FreeBSD, Mandriva, Apple), what would simplify your build system?

My toolchain distribution, llvm-mingw, already uses compiler-rt builtins as the deafult rtlib, and I don’t have any issues with it - I have no requests to make that would make things easier. In fact, all the things suggested here would mainly lead to churn in needing to change an already working setup regarding how things are built.

@rengolin

For Win/GNU-based environments, libgcc is still there. Are there any tools, code size, special requirements that stop you linking compiler-rt by default with Clang on your system?

This is mostly a matter of toolchain setup. My llvm-mingw distribution uses compiler-rt exclusively. The MSYS2 distribution contains a number of different environments with different setups. Some environments use clang as default compiler, and use compiler-rt as default rtlib. Other environments use GCC (and libstdc++) as default, and in those environments, Clang also uses libgcc as default rtlib. Using compiler-rt instead of libgcc there might work, but could lead to subtle issues. (Those environments use emulated TLS, where I believe a shared linked libgcc handles the global state of it. Using compiler-rt there would lead to some DLLs using one state and others using another. The same goes for runtime-registered unwind info - although that’s about libunwind vs libgcc, not compiler-rt.)

@rengolin

If so, would linking just the shim delta (ex. 128-bit) as a separate library name be a reasonable alternative solution?

This might be helpful in some cases. It doesn’t make any actual difference in the cases I’ve mentioned before though (e.g. clang-cl and 128-bit, where the main issue is making sure that the compiler_rt builtins is available at all, and making sure that the linker does include it).

[rengolin]

@mstorsjo

I’m not sure what more there is to gain. From a clarity point of view, things aren’t necessarily ideal, but I don’t see how changing it would be worth the churn for everybody who currently are building it. (If they’re moved out, it’d break the build scripts for anyone assembling a toolchain, expecting to be able to build it like it is now. Thus, moving it probably requires some temporary glue to let the old way of building it still work for a transition period.)

Right. The split doesn’t need to be “physical” (another base dir) but build system-wise, as you mention.

Enough people in this thread has trouble building just the builtins, so perhaps the “runtimes” build isn’t quite obvious to the community at large. Last time I asked, not many people seemed to know what the status of that build was and how to make it work.

Perhaps the solution to this problem is just to make it more clear, or make it the default build type for compiler-rt, and slowly deprecate the old “projects” build. Also possibly add a small separation for the delta helpers (ex 128-bit) as a separate library if requested and add that to clang by default.

@mstorsjo

This still probably isn’t trivial and some didn’t quite like it (it needs to add a per-module flag and tracked properly for e.g. LTO), but I feel this point is discarding the suggestion entirely by making it one magnitude harder than it should be.

You’re right, I apologise. I have waved too large a hand over the proposal. I still think that this is fraught with propagation issues, but I’m hand-waving quite hard here, so don’t take my pessimism as a data point.

@mstorsjo

(Those environments use emulated TLS, where I believe a shared linked libgcc handles the global state of it. Using compiler-rt there would lead to some DLLs using one state and others using another. The same goes for runtime-registered unwind info - although that’s about libunwind vs libgcc, not compiler-rt.)

Thanks! Those are some of the issues I remember hearing about but couldn’t quite remember the details. I agree it’s a toolchain setup issue, but it can be substantial issues that make it hard to hack around. I wouldn’t like to make that worse.

[mstorsjo]

@rengolin

Right. The split doesn’t need to be “physical” (another base dir) but build system-wise, as you mention.

Enough people in this thread has trouble building just the builtins, so perhaps the “runtimes” build isn’t quite obvious to the community at large. Last time I asked, not many people seemed to know what the status of that build was and how to make it work.

Perhaps the solution to this problem is just to make it more clear, or make it the default build type for compiler-rt, and slowly deprecate the old “projects” build.

Yes, this certainly is true. Compiler-rt has got a lot of custom code on its own with the purpose of somewhat emulating the “cross”/multilib aspects that the runtimes build is all about doing explicitly (so that within the build of the runtimes, it’s a proper cmake cross build) - and iirc @delcypher (and @petrhosek) has spent some effort on unwinding some of this.

Personally I don’t really use the runtimes build at all - I build the runtimes separately manually by doing separate cmake builds for the runtimes (starting out with compiler-rt/lib/builtins, then building libunwind/libcxxabi/libcxx) where I manually configure the cmake builds as cross/multilib builds.

Requiring compiler-rt to be built as a runtime like the others probably simplifies some things here. But another argument in the other direction, that I’ve heard, is that for some builds, users aren’t even building Clang at all, so there’s no freshly-built Clang to use for building the runtimes. Such users use a preexisting GCC to build LLVM and compiler-rt, and a different language frontend on top of this (I don’t remember if this was Rust or some other one). If compiler-rt isn’t allowed as part of LLVM_ENABLE_PROJECTS, they’d need to do two separate cmake builds - one pointing at llvm-project/llvm and one pointing at llvm-project/compiler-rt/lib/builtins (or llvm-project/compiler-rt or llvm-project/runtimes).

That usecase needs to be supported somehow, but if we make compiler-rt behave more strictly like the other runtimes, it probably will need to be either as part of the runtimes build (with a Clang build) or with two separate cmake invocations.

@rengolin

Also possibly add a small separation for the delta helpers (ex 128-bit) as a separate library if requested and add that to clang by default.

Yes, possibly. I’m not totally sold on this approach but I’m not strictly opposed either.

@rengolin

@mstorsjo

This still probably isn’t trivial and some didn’t quite like it (it needs to add a per-module flag and tracked properly for e.g. LTO), but I feel this point is discarding the suggestion entirely by making it one magnitude harder than it should be.

You’re right, I apologise. I have waved too large a hand over the proposal. I still think that this is fraught with propagation issues, but I’m hand-waving quite hard here, so don’t take my pessimism as a data point.

Ok, thanks for the honest acknowledgement!

@rengolin

@mstorsjo

(Those environments use emulated TLS, where I believe a shared linked libgcc handles the global state of it. Using compiler-rt there would lead to some DLLs using one state and others using another. The same goes for runtime-registered unwind info - although that’s about libunwind vs libgcc, not compiler-rt.)

Thanks! Those are some of the issues I remember hearing about but couldn’t quite remember the details. I agree it’s a toolchain setup issue, but it can be substantial issues that make it hard to hack around. I wouldn’t like to make that worse.

Yeah; the issue is mostly that libgcc has many roles - some of these roles track global state, where you really do want to use the same shared libgcc as the rest of your toolchain stack. Others are just simple helper functions, where a static library with possibly overlapping functions should be mostly uncontroversial (and this probably is mostly what people refer to when considering “builtins”). There were some valid cases brought up where mixing and matching isn’t ideal though - atomics and potential issues with functions returning differently rounded results.

[efriedma-quic]

On some targets, clang is trying to be a drop-in replacement for a non-clang system compiler. For example Linux glibc, or MSVC. Often, this means clang ends up being used in ways where it doesn’t control the link line. On Linux, the user might want to build a static library, and allow users to link against it using gcc. With MSVC, on top of that, build systems often invoke the linker directly. So it’s impossible to link compiler-rt.builtins or any successor library “by default”.

Assuming we want to continue supporting such configurations, our only choice is to embed any helpers which aren’t provided by the platform into each object file.

[mstorsjo]

@efriedma-quic

On some targets, clang is trying to be a drop-in replacement for a non-clang system compiler. For example Linux glibc, or MSVC. Often, this means clang ends up being used in ways where it doesn’t control the link line. On Linux, the user might want to build a static library, and allow users to link against it using gcc.

Ok, that’s a very good point - can’t really argue against that.

@efriedma-quic

With MSVC, on top of that, build systems often invoke the linker directly. So it’s impossible to link compiler-rt.builtins or any successor library “by default”.

In MSVC environments, it’s possible to embed linker directives into the object files, so it’s somewhat possible there, though. And MSVC does ship with clang_rt.builtins-*.lib as part of the MSVC lib directory (for x86 and x64 only though).

@efriedma-quic

Assuming we want to continue supporting such configurations, our only choice is to embed any helpers which aren’t provided by the platform into each object file.

No objections from me wrt doing that.

The case primarily discussed here, RTLIB::MULO_I128, is one where the codegen seems able to generate inline code if the libcall is unavailable - and I guess the main tradeoff is between having the code expanded inline vs calling a rtlib function. But for e.g. RTLIB::SDIV_I128 and the related ones, the codegen layer doesn’t seem to handle it missing at all.

For emitting an embedded helper, I guess there’s not much difference between those cases though - we’d need to carry a canned version of such a function to emit when necessary?

[tyker]

maybe a stupid question but why would the compiler not link with the builtin it used. and let the linker cleanup duplicates ?

[nickdesaulniers]

Thoughts on inferring the rtlib from the triple, then checking that?

https://reviews.llvm.org/D143120

The logic for inferring is incomplete, and doesn’t address what we should do if the user sets --rtlib, but seems pretty simple to me. Thoughts?

[JonChesterfield]

TLDR, lets use indirection to fix this. Reasoning followed by implementation proposal follows.

There is no intrinsic difference between code in a compiler runtime that we link at a given point in time vs generating the same function on the fly in the compiler. Some things we lower need to turn into a large number of instructions, a function is a convenient way to encapsulate that, a function written in a library is easier to test than one the compiler creates on demand.

We have two compiler-builtin libraries, the gcc one and our own, where users might choose either. For that to work, we either need to encode which builtin library is in use before choosing to rely on functions in it, or to have the same set of functions in each. Specifying link time information at compile time is difficult, in part because users don’t currently do that and may not want to.

We extended our builtin library because that was more convenient than embedding the code in our compiler and have not extended the gcc one to match. Thus, bang, with the workaround of telling people to link against our library,

The solutions are thus:

• extend gcc’s lib
• move the llvm extensions out of compiler-rt
• change codegen to not use the llvm extensions any more

It seems pretty ugly to have two compiler builtin libraries where the controlling distinction for which one contains a given function is whether gcc’s builtins has the same function. Also somewhat fragile if some of our extensions end up being picked up by gcc later.

I’d like to suggest we have one library, a static archive of machine code much as we have today. (I’d like the builtins to be available as IR, but that’s a side point). Prefix everything in it with llvm_. In addition to that, we have a separate entity that contains a list of all the symbols exported by gcc’s builtins defined as aliases to symbols in said static archive. We emit calls to symbols with the same names they have today, that is no llvm_ prefix, except for bugfixes where functions that don’t exist in the gcc libs change to call to the llvm_ prefixed ones.

On linking, we either, as the user requested:

• link gcc’s builtins followed by our compiler builtins machine code
• link our file-of-aliases followed by our compiler builtins machine code

In the first case, all the symbols that exist in gcc’s builtins lib will resolve to that, as requested, and remaining unresolved symbols will resolve to ours. We have control over whether our compiler builtins should use symbols from gcc or our own based on whether we write calls to llvm_foo or foo.

In the second case, all the symbols we emitted are resolved first to the prefixed function via the alias and then to the corresponding function in our compiler builtins. Minor link time overhead.

What we get in exchange for writing out the list of symbols in compiler-rt (via nm on gcc’s one if we like) and teaching the driver to link llvm-compiler-builtin.a immediately after the gcc-or-our builtin lib:

• Preserves machine code linking semantics
• Allows exact compatibility with gcc
• Can evolve it if gcc adds functions that do the same thing as ours with the same name
• Can also deal with gcc adding symbol collisions with different calling convention or semantics
• Requires no change to the llvm backends
• We can link to functions in the llvm lib or the gcc-or-llvm one by prefixing llvm_ or not
• Leaves us free to continue adding whatever functions are convenient to compiler-rt builtins
• We can continue to test through the gcc names or change to testing our own ones

Whether we move the result to a new top level or not is somewhat orthogonal. Likewise whether the archive containing machine code gets the current name or whether the file of symbol aliases does.

Thanks for reading!

[efriedma-quic]

@mstorsjo

@efriedma-quic

With MSVC, on top of that, build systems often invoke the linker directly. So it’s impossible to link compiler-rt.builtins or any successor library “by default”.

In MSVC environments, it’s possible to embed linker directives into the object files, so it’s somewhat possible there, though. And MSVC does ship with clang_rt.builtins-*.lib as part of the MSVC lib directory (for x86 and x64 only though).

There are two ways you can embed a linker directive:

• As an absolute path. Then you’re can’t link on a different machine.
• Just a filename. Then you pick up whatever clang_rt.builtins is on the libpath… which is probably the MSVC one. This solves the issue with multiply specifically, but if we ever try to add a new function to compiler-rt.builtins, it still won’t be available for a few years.

@tyker

maybe a stupid question but why would the compiler not link with the builtin it used. and let the linker cleanup duplicates ?

A few people have suggested this; see previous discussion of “COMDAT”, embedding the implementation, etc.

[smeenai]

Might have missed this mentioned somewhere already, but -rtlib= already affects codegen in that AArch64 out-of-line atomics are unconditionally used for compiler-rt, and only used for libgcc above a certain version. See llvm-project/Linux.cpp at 7bc9dc26bc9beb159547f7160d852543bc555552 · llvm/llvm-project · GitHub and llvm-project/Clang.cpp at 7bc9dc26bc9beb159547f7160d852543bc555552 · llvm/llvm-project · GitHub

[rengolin]

Thanks everyone for the varied points of view!

@efriedma-quic

On some targets, clang is trying to be a drop-in replacement for a non-clang system compiler. For example Linux glibc, or MSVC. Often, this means clang ends up being used in ways where it doesn’t control the link line. On Linux, the user might want to build a static library, and allow users to link against it using gcc. With MSVC, on top of that, build systems often invoke the linker directly. So it’s impossible to link compiler-rt.builtins or any successor library “by default”.

I think this summarises the whole problem in a very neat paragraph. There’s no way we can control what we can or cannot use as runtime library for any other compiler at arbitrary versions.

Solutions like COMDAT, build attributes and propagating choices down the compilation path can work in some cases but not others. However, all of them fail when handling expectations from external toolchains.

Unless the Clang/LLVM community takes the radical decision that we don’t want to emulate those native toolchains, and stop being a drop-in replacement, we have no other choice. And even if that does happen at some point, this thread isn’t about that, but about the immediate effect of emitting symbols that other runtime libraries cannot (and will not) honour.

@efriedma-quic

Assuming we want to continue supporting such configurations, our only choice is to embed any helpers which aren’t provided by the platform into each object file.

As much as I don’t like this solution, I cannot see any other way forward.

Assuming this is as much a consensus as I think it is, what are the next steps to make this happen?

Some questions that may help the planning:

• Do we split the sources or just the build?
• How do we guarantee these builtins always get compiled without ENABLE_PROJECTS?
• How would this complicate the build systems (plural)?
• How would this affect cross-compilation?
• What if some other toolchain adds the same symbol on their runtimes in the future?

But at least, there are some benefits:

• It shouldn’t affect existing RT environments
• It’ll be backward compatible with all toolchains
• If we’re careful with symbol naming and linkage types, it will also be forward compatible

[tschuett]

If you want to separate the build system support (cmake, meson, …) from clang needs, then you could install the builtins as bitcode in the resource directory. All builtins get a special prefix. IDK llvm_ . Then clang can just the load the builtins that it needs into the current process and decide whether to inline or make it a call. The compiler-rt is still available for the linker.

[rengolin]

@tschuett

Then clang can just the load the builtins that it needs into the current process and decide whether to inline or make it a call.

The problem isn’t when Clang generates the builtins, but LLVM. This is why the “fix” is to tell LLVM those builtins aren’t legal.

In Clang, you can always check __has_builtin, but when LLVM lowers to the calls, there is no control over what gets generated, other than marking it completely illegal.

[vtjnash]

@rengolin

Solutions like COMDAT, build attributes and propagating choices down the compilation path can work in some cases but not others. However, all of them fail when handling expectations from external toolchains.

Unless the Clang/LLVM community takes the radical decision that we don’t want to emulate those native toolchains, and stop being a drop-in replacement, we have no other choice. And even if that does happen at some point, this thread isn’t about that, but about the immediate effect of emitting symbols that other runtime libraries cannot (and will not) honour.

Even when those fail, this sound like it could be an improvement over the current situation, where some things just fail, and there is no straightforward way to fix it. An example of that is 128 integers on 32-bit machines. LLVM has support for them, but expects the user to provide some helper functions. Implementations of those functions are in compiler-rt, but not in gcc or msvcrt. Thus when using clang as a drop-in replacement for gcc in existing projects, those would hopefully continue to function unchanged. But for projects that export features not available in gcc (especially other languages that are built on llvm, such as rust and julia), then just improving the ability to get those pieces from compiler-rt, even just when clang is the linker driver, I think would be a beneficial improvement!

@rengolin

@efriedma-quic

Assuming we want to continue supporting such configurations, our only choice is to embed any helpers which aren’t provided by the platform into each object file.

Just as an observation, I think this is essentially what the NVPTX backend does, via libdevice:
https://llvm.org/docs/NVPTXUsage.html#linking-with-libdevice
https://docs.nvidia.com/cuda/libdevice-users-guide/basic-usage.html

@rengolin

• How would this affect cross-compilation?

It seems like it would require having a .bc and/or .a file for every possible combination of target platform ABIs? (that is what libdevice does)

@rengolin

• What if some other toolchain adds the same symbol on their runtimes in the future?

I think there was a suggestion above to name the function with a prefix __llvm_, so it won’t get added in the future externally, and then to re-export it as a weak symbol so that the definitive name can be replaced by a strong definition provided later by the platform (hoping just that the ABIs are compatible, since they should hopefully be aware of the existing LLVM usage)

[rengolin]

@vtjnash

Even when those fail, this sound like it could be an improvement over the current situation, where some things just fail, and there is no straightforward way to fix it. An example of that is 128 integers on 32-bit machines. […]

Your example is the main case in the issue reported, the code review and the whole discussion here. We’re not looking for “an improvement”, we’re looking for a solution that doesn’t create more problems, and all of those do. In that sense, none of those would improve the situation, they’d just move problems around.

Also, the “questions” were rhetorical. Just a list of things we need to think about when proposing alternatives. Ideas that answer some but not all of those questions are not helpful as they fall into the caveat above.

I’m also not claiming those are all the requirements, I’m sure there’s more to answer than just those. My point was just that we need to cover all bases on any proposed solution.

[efriedma-quic]

@rengolin

• Do we split the sources or just the build?
• How do we guarantee these builtins always get compiled without ENABLE_PROJECTS?
• How would this complicate the build systems (plural)?
• How would this affect cross-compilation?

The simplest way to “embed” functions is probably to generate IR for them on demand, along the lines of llvm/lib/Transforms/Utils/IntegerDivision.cpp . I think that avoids all the tricky bits related to cross-compilation/build systems/ABI/etc.: there is no separate build step at all, and we don’t need any external files.

Downside is the amount of work involved for each routine; we’d basically need to reimplement the routines we need, and writing large routines using IRBuilder is a bit painful. Maybe we could come up with some tooling to make it easier.

@rengolin

What if some other toolchain adds the same symbol on their runtimes in the future?

If we’re inventing names, we should prefix them with something that nobody will use by accident (something along the lines of “__llvm_builtin_muloverflow64_v1”). We sort of encroached on gcc’s namespace with the overflow routines; we should avoid that in the future.

Or do you mean if the target adds new routines with the functionality we need, and we decide to use them? We already do some form of version specification/detection for many targets (Android API versions, MSVC versions, etc.).

[rengolin]

@efriedma-quic

The simplest way to “embed” functions is probably to generate IR for them on demand, along the lines of llvm/lib/Transforms/Utils/IntegerDivision.cpp . I think that avoids all the tricky bits related to cross-compilation/build systems/ABI/etc.: there is no separate build step at all, and we don’t need any external files.

I like this idea. I imagine we can make that pass only do something if the target doesn’t have the libcall legal, so that, if other runtimes implement a locally-better version in the future, we can just make it legal and get the runtime calls again.

This pretty much fixes all of the problems in one go.

@efriedma-quic

Downside is the amount of work involved for each routine; we’d basically need to reimplement the routines we need, and writing large routines using IRBuilder is a bit painful. Maybe we could come up with some tooling to make it easier.

It’s a mechanical change for a small number of routines. We can worry about how to make it easier later, but to be honest, looking at the IRBuilder sequence and what the asm file would be, it’s not that different.

@efriedma-quic

Or do you mean if the target adds new routines with the functionality we need, and we decide to use them? We already do some form of version specification/detection for many targets (Android API versions, MSVC versions, etc.).

This is what I meant, yes. Depending on the answers above, this will have to be different.

Build hacks, alternate local symbols and similar solutions would have to worry about implementing every possible version (and wrappers, see gnu vs eabi divrem on Arm).

But if we just do library-as-code-gen, then it’s a matter of making the builtin legal and we wouldn’t have to do anything else.

[poeplau]

@rengolin

@efriedma-quic

The simplest way to “embed” functions is probably to generate IR for them on demand, along the lines of llvm/lib/Transforms/Utils/IntegerDivision.cpp . I think that avoids all the tricky bits related to cross-compilation/build systems/ABI/etc.: there is no separate build step at all, and we don’t need any external files.

I like this idea. I imagine we can make that pass only do something if the target doesn’t have the libcall legal, so that, if other runtimes implement a locally-better version in the future, we can just make it legal and get the runtime calls again.

This pretty much fixes all of the problems in one go.

Isn’t that what’s already happening for the multiplication with overflow check in the original issue? If the lib call isn’t available (e.g., in the various backends that have already disabled it), it’s replaced with an inline implementation at the level of the SelectionDAG: https://github.com/llvm/llvm-project/blob/4901245481b6588056638b8fc38598615039314a/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp#L4539.

[rengolin]

@poeplau

Isn’t that what’s already happening for the multiplication with overflow check in the original issue?

Not quite.

The “fix” involves generating generic code, which relies on type legalization (as you pointed out), which is the most naive code we could possibly produce. The IntegerDivision file adds crafted code-generation for specific-purpose routines, just like a library would.

In a way, that’s like inlining a library call into the code at compile time, with the library written directly in IR. It’s similar to custom lowering, which is another option, but further away from library writers’ world.

Compared to an external IR file, this is in the compiler, so tracked with the rest of code-gen, and doesn’t have to be stored somewhere, merged, inlined. Compared to libraries, this doesn’t need LTO to inline code.

The bad part is writing “assembly code” as a series of IRBuilder calls, which can be cumbersome for library writers (who usually prefer assembly). But being just a few small things, and similar enough to asm, I think it’s manageable.