We at Azul Systems are working on a multi-threaded LLVM based compiler. It can run several compilations each of which compiles its own module in its own thread.
One of the limitation we face is that all threads use the same options (instances of cl::opt). In other words, the options are global and cannot be changed for one thread and left unchanged for the others.
+Lang Hames since he’s playing with multithreaded compilation in the ORC JIT too.
My off-the-cuff thought (which is a lot of work, I realize) would be that cl::opts that aren’t either in drivers (like opt.cpp, llc.cpp, etc) or developer options (dump-after-all, things like that) shouldn’t be cl::opts and should be migrated to options structs and the like?
I realize that’s a ton of work, and we all sort of cringe a little when we add another “backend option” (accessing cl::opts via -backend-option in the Clang driver when invoking clang cc1) & then do it anyway, etc… but would be pretty great to clean it up and have a clear line about what cl::opts are for.
(totally reasonable for you to push back and say “that’s not the hill I want to die on today”, etc - and see what everyone else thinks)
+Lang Hames <mailto:lhames@gmail.com> since he's playing with multithreaded compilation in the ORC JIT too.
One nit about terminology - there are two different flavors of "multithreaded compilation".
Some people read it as "doing parallel processing of a single compilation job" and some as
"doing parallel independent compilation jobs".
Azul's Falcon JIT compiler does the latter.
My off-the-cuff thought (which is a lot of work, I realize) would be that cl::opts that aren't either in drivers (like opt.cpp, llc.cpp, etc) or developer options (dump-after-all, things like that) shouldn't be cl::opts and should be migrated to options structs and the like?
+1
It would be great to have a direct API accessing/setting up these "option structs" for in-process JIT clients
that start many different compilations.
Having to parse option strings has always striked me as something rather clumsy.
On other hand, ability to replay compilation with standalone opt and still have the same controls over functionality of optimizer
happens to be a great time saver. Thus having a way to control these non-cl::opt things from opt's command-line is also
a good thing to have.
(something along the line of a difference between legacy PM's command-line pass interface - where every pass presents itself as an option,
and new PM's -passes= single option).
One nit about terminology - there are two different flavors of
“multithreaded compilation”.
Some people read it as “doing parallel processing of a single
compilation job” and some as
“doing parallel independent compilation jobs”.
Azul’s Falcon JIT compiler does the latter.
ORC is also doing parallel independent compilation jobs, so this would be a win for the ORC APIs too.
This would be a very welcome feature for Mesa as well, thank you for doing this!
When several threads compile different modules the compiler options > (instances of cl::opt) cannot be set individually for each thread.
That > is because the options are visible to all threads. In other words all > options are global.> > It would be convenient if the options were specific to LLVMContext and > they were accessed through an instance of LLVMContext. This kind of > change would need changes in all source files where options are used.> > This patch proposes a solution that needs minimal changes in LLVM source > base.> > It is proposed to have a thread local set of re-defined option values > mapped by pointers to options.
That seems very sensible and pragmatic to me.
Specifically, every time a program gets/sets a value for an option it is checked if the current thread local context is set for the current thread and the option has its local copy in this context. If so the local copy of the option is accessed, otherwise the global option is accessed. For all programs that existed so far the context is not set and they work with the global options. For new multi-threaded compilers (where every thread compiles its own module) every thread can be linked to its own context (see ContextValues) where any option can have its thread specific value that do not affect the other threads' option values. See the thread_routine() in the test ContextSpecificValues2.
Just to make sure I'm understanding this correctly: When a thread has a non-null option context, then options set in the global context will have no effect at all for that thread. Right? (That's what I would expect and what makes sense to me.)
+Lang Hames mailto:[lhames@gmail.com](mailto:lhames@gmail.com) since he’s playing with
multithreaded compilation in the ORC JIT too.
One nit about terminology - there are two different flavors of
“multithreaded compilation”.
Some people read it as “doing parallel processing of a single
compilation job” and some as
“doing parallel independent compilation jobs”.
Azul’s Falcon JIT compiler does the latter.
My off-the-cuff thought (which is a lot of work, I realize) would be
that cl::opts that aren’t either in drivers (like opt.cpp, llc.cpp,
etc) or developer options (dump-after-all, things like that) shouldn’t
be cl::opts and should be migrated to options structs and the like?
+1
It would be great to have a direct API accessing/setting up these
“option structs” for in-process JIT clients
that start many different compilations.
Having to parse option strings has always striked me as something rather
clumsy.
On other hand, ability to replay compilation with standalone opt and
still have the same controls over functionality of optimizer
happens to be a great time saver. Thus having a way to control these
non-cl::opt things from opt’s command-line is also
a good thing to have.
Oh, sure - that’s true of lots of config options passed through structs today & I believe would/should continue to be true as these values are migrated. That’s necessary for testing those configuration options from within LLVM lit tests as we usually do.
As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.
We also have a system for options in LLVMContext (see http://llvm.org/219854) that unfortunately was only ever used for a single options and was not followed through to be used for all the other options we have…
As another data point: I’ve been working on a multi-threaded compiler using LLVM in the past and I have patched passes to accommodate this, for instance by taking their parameters in a constructor, and only defaulting to the cl::opt.
The consensus at the time was that the cl::opt were only here for debugging / overriding default, but should not be the way any user of “LLVM as a library” set the options.
Prime motivation for our JIT is a desire to have a tight control on behavior of individual compilation
while leaving other compilations work in “default” mode.
One major use is indeed debugging, say, opt-bisect for a specific method compilation,
or print-before/after for a specific method/pass.
However, we also have other ideas on how to use this - say, we would like to have different “optimization levels”,
and for that we need to tweak various defaults - thresholds or off/on switches - that select exact amount of
work for optimizer. In some cases we can handle that through configuring a pass during its construction time,
but for most cases there are no such controls.
Similarly we would use command line optimizer tweaks in order to guide performance analysis, but that is kinda
on border with debugging.
As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.
The toughest problems arise for us when Mesa (i.e., OpenGL drivers that use LLVM as a shader compiler backend) is used inside an application that itself uses LLVM.
The application may be setting options in LLVM for whatever reason, which then affects compilation of shaders in Mesa. That's a pretty Bad Thing.
As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.
The toughest problems arise for us when Mesa (i.e., OpenGL drivers that use LLVM as a shader compiler backend) is used inside an application that itself uses LLVM.
The application may be setting options in LLVM for whatever reason, which then affects compilation of shaders in Mesa. That's a pretty Bad Thing.
I would imagine that you need to completely hide your instance of LLVM via some linker magic (visibility etc).
Otherwise it sounds like a nightmare to manage.
What if LLVMs are not the same?
This problem seems to be much harder to solve than our JIT one (where we are the owners/only users of LLVM instance).
As I just noted in the review: I wonder about the motivation for this, if we find that cl::opts are not just used as debug flags for users, then we really should rather find ways to expose proper APIs through things like TargetOptions.h or function/module attributes. It would certainly help the discussion if you could describe what motivated you to do the patch in the first place.
The toughest problems arise for us when Mesa (i.e., OpenGL drivers that use LLVM as a shader compiler backend) is used inside an application that itself uses LLVM.
The application may be setting options in LLVM for whatever reason, which then affects compilation of shaders in Mesa. That's a pretty Bad Thing.
I would imagine that you need to completely hide your instance of LLVM via some linker magic (visibility etc).
Otherwise it sounds like a nightmare to manage.
What if LLVMs are not the same?
This problem seems to be much harder to solve than our JIT one (where we are the owners/only users of LLVM instance).
We can do static linking of LLVM for our own releases, but this doesn't work for Mesa releases in Linux distributions.
Linux distributions prefer dynamic linking for their own good reasons, and also ensure that the same LLVM binary will be used. So there's no compatibility issue on that front, but the issue of isolating the data / options used by the two instances remains.
Linux distributions prefer dynamic linking for their own good reasons,
and also ensure that the same LLVM binary will be used. So there’s no
compatibility issue on that front, but the issue of isolating the data /
options used by the two instances remains.
Would adding a thread-local mode to ManagedStatic help? There could also be a ManagedStatic context, similar to an OpenGL context in that it can be bound to a set of threads.
Yes, but the global context has no effect on your thread only for those options that have been changed in this thread’s context.
That is because a thread local copy of an option is created in the current thread context when this option is set by this thread or another thread with the same option context. If the option has not been set then its global value is used.
For your case, I would suggest that you explicitly set a new thread option context for every thread (or thread group) that uses LLVM. This way the threads/groups will not affect each other and will use the options in the copy-on-write way.
May be it makes sense to try playing with the tests in the patch to better understand the use model.
The idea of having a separate API for options in LLVMContext is good but difficult to implement. That is probably why it has not been evolved.
There are so many cl::opt all over the code to change … This would also split all cl::opt into incompatible groups: those that are bound to LLVMContext and those that are not.
With the proposed thread local option context (https://reviews.llvm.org/D53424) your idea can be simulated by binding LLVMContexts with ContextValues and setting threads to their LLVMContexts’ ContextValues.
I believe that D53424 is a minimal change that can greatly extend the cl::opt-based configuration flexibility without affecting other aspects. This change does not contradict with module flags, LLVMContext Debug options and other ways we have to customize the pipeline.
Just to make sure I'm understanding this correctly: When a thread has a
non-null option context, then options set in the global context will
have no effect at all for that thread. Right? (That's what I would
expect and what makes sense to me.)
Yes, but the global context has no effect on your thread only for those options that have been changed in this thread's context.
That is because a thread local copy of an option is created in the current thread context when this option is set by this thread or another thread with the same option context. If the option has not been set then its global value is used.
For your case, I would suggest that you explicitly set a new thread option context for every thread (or thread group) that uses LLVM. This way the threads/groups will not affect each other and will use the options in the copy-on-write way.
That's not quite what we'd need, and we can't control all threads -- Mesa is a library, and so we can't control what other threads are doing.
That said, it may be possible to add what we need on top of what you're proposing without too much effort. Basically, we'd need the ThreadOptionContext to be optionally "opaque" in the sense of hiding global options.
Do you actually need the ThreadOptionContext to be "transparent" for your use case, or is that just an accidental consequence of the current implementation?
Linux distributions prefer dynamic linking for their own good reasons,
and also ensure that the same LLVM binary will be used. So there's no
compatibility issue on that front, but the issue of isolating the
data /
options used by the two instances remains.
Would adding a thread-local mode to ManagedStatic help? There could also be a ManagedStatic context, similar to an OpenGL context in that it can be bound to a set of threads.
I don't think most ManagedStatics actually need it. Skimming over it, the only that sticks out is the CurrentDebugType, but since it's only for debugging it doesn't seem too critical.
If all threads were bound to a non-null context then there would be no one which initialized the global options. This would result in unset default values. That is why we need the very first thread (that is used to load and initialize LLVM libraries) to have the ThreadOptionContext unset, so the default option values get into the default option storage. Then, I believe, it would be ok to force all threads to have non-null ThreadOptionContexts. In other words we need to identify the library loading thread and the initialization time period while the ThreadOptionContext must be null.
These are just my thoughts. I have not tried this use model.
I made the ThreadOptionContext “transparent” to not impose constraints that are not needed for the implementation.