Marking a function prototype as being "persistent"

Project Infrastructure LLVM Dev List Archives
1

I'm using LLVM to JIT-compile the XL programming language. I've recently added a whole-program optimization phase that gives excellent results, but I noticed that the StripDeadPrototypesPass was removing all references to my runtime support functions.

why is that a problem?

Because I have pointers to functions which suddenly go bad.

Apparently, this depends on the oddly-named UnitAtATime parameter. Set it to false. Try again. Suggestion: what about a comment explaining what "UnitAtATime" means :slight_smile:

If UnitAtATime is false this means that you are optimizing functions as you are
generating them, rather than first generating all functions and other globals
and only then optimizing. It's basically a historical anachronism coming from
the way GCC used to work.

Thanks for explaining.

Why not redefine it from historical anachronism to architectural flag, that indicates you may generate more code with the same functions and globals later? And then, you could pass the same flag to createStandardLTOPasses, and that would skip the GlobalDCE passes.

Even then, my runtime function prorotypes are still being stripped away by the GlobalDCE added by createStandardLTOPasses.

Sure, and why not? Unused prototypes are not used for anything, they won't
turn up in the generated code for example.

They are not used this time. But why can't I generate additional code in the same module after having run LTO?

2

Hi Christophe,

Apparently, this depends on the oddly-named UnitAtATime parameter. Set it to false. Try again. Suggestion: what about a comment explaining what "UnitAtATime" means :slight_smile:

If UnitAtATime is false this means that you are optimizing functions as you are
generating them, rather than first generating all functions and other globals
and only then optimizing. It's basically a historical anachronism coming from
the way GCC used to work.

Thanks for explaining.

Why not redefine it from historical anachronism to architectural flag, that indicates you may generate more code with the same functions and globals later? And then, you could pass the same flag to createStandardLTOPasses, and that would skip the GlobalDCE passes.

LTO is for doing optimizations that are only valid when the module contains
everything that is needed to build the final executable. So adding a flag to
say "not everything is there after all" makes no sense to me.

Even then, my runtime function prorotypes are still being stripped away by the GlobalDCE added by createStandardLTOPasses.

Sure, and why not? Unused prototypes are not used for anything, they won't
turn up in the generated code for example.

They are not used this time. But why can't I generate additional code in the same module after having run LTO?

Because by definition LTO is for when everything is there! Why don't you
create your own sequence of passes that does what you want rather than calling
createStandardLTOPasses? One of the nice features of LLVM is that you can
run any passes you choose in any order you like. That said, probably the reason
you are getting improved optimization is due to running the "internalize" pass.
If so be warned that it is not safe to run internalize unless all functions and
their bodies are present in the module. If you run internalize, optimize, and
then later add a few more functions to the module all kinds of nasty subtle
things can go wrong, not just those you saw with GlobalDCE.

Ciao, Duncan.

3

Hi Duncan,

LTO is for doing optimizations that are only valid when the module contains
everything that is needed to build the final executable. So adding a flag to
say "not everything is there after all" makes no sense to me.

And indeed, everything is there when I call LTO. The flag is not "not everything is there after all", but "don't damage your inputs".

Even then, my runtime function prorotypes are still being stripped away by the GlobalDCE added by createStandardLTOPasses.

Sure, and why not? Unused prototypes are not used for anything, they won't
turn up in the generated code for example.

They are not used this time. But why can't I generate additional code in the same module after having run LTO?

Because by definition LTO is for when everything is there!

And again, everything is there. I just want to be able to reuse the same inputs later for incremental recompilation.

Why don't you create your own sequence of passes that does what you want rather than calling createStandardLTOPasses?

As I wrote in an earlier reply, been there, done that. That forces me to maintain a quasi-clone of standard passes.

One of the nice features of LLVM is that you can run any passes you choose in any order you like.

Having standard LTO passes get in the way of incremental recompilation is not a feature.

That said, probably the reason you are getting improved optimization is due to running the "internalize" pass. If so be warned that it is not safe to run internalize unless all functions and their bodies are present in the module.

All functions and bodies are present, so that's not the issue. The issue is that some inputs of LTO are destroyed as a side effect of running it.

If you run internalize, optimize, and then later add a few more functions to the module all kinds of nasty subtle things can go wrong, not just those you saw with GlobalDCE.

Do you mean I should assume for example that the Functions I generated are also irreversibly damaged? If so, what passes do I need to avoid if I want to preserve the ability to do incremental recompilation?

Thanks for your help
Christophe

4

Why don't you create your own sequence of passes that does what you want rather than calling createStandardLTOPasses?

As I wrote in an earlier reply, been there, done that. That forces me to maintain a quasi-clone of standard passes.

No, this is a different suggestion, it is to maintain a separate list
of what passes are known to not violate your assumptions, ie don't run
GlobalDCE and StripDeadPrototypes, and maybe some others. I don't
know enough to give you a list. This could be better documented. I
know we ran into issues using the standard IPO passes for unladen
swallow, but I don't know what the resolution was.

One of the nice features of LLVM is that you can run any passes you choose in any order you like.

Having standard LTO passes get in the way of incremental recompilation is not a feature.

I would say that deleting as much dead code as possible during LTO is a feature.

Reid