Making it possible to clear the LLVMContext

Hello,

I’m trying to develop a way to reliably clean the LLVMContext in order to make it possible to use it multiple times.

LLVMContext itself is an almost empty object delegating almost all its content to LLVMContextImpl.
This makes it very clean ideally, because clearing the LLVMContext would be as easy as deleting the LLVMContextImpl and creating a new one.

The problem is that for some reason which I’m not aware of LLVMContextImpl is actually exposed as a public pointer in the LLVMContext interface,making it publicly available to objects that use it directly (this seems to happen quite a lot in the codebase).

In LLVMContext the LLVMContextImpl is contained in a pImpl pointer that is const (the pointer itself can’t be changed) and I guess this is some kind of protection against object replacing the LLVMContextImpl directly, which stops us from just deleting it + getting a new one.
In addition to that, being pImpl public, there is no guarantee that objects don’t rely on pImpl remaining always the same pointer.

This makes it more difficult to clear LLVMContext.

An approach I thought of could be adding a clear() method to LLVMContext that:

  • Calls directly the destructor of LLVMContextImpl on the pImpl object
  • Uses a placement new to reinitialize the object.
  • Recreates the fixed metadata kinds like the LLVMContext constructor does

I’m attaching a patch that show this approach in this mail.

I would like to know a general idea about what people think about this and see what people think would be the best approach would be.

Thanks,
Marcello

clear_llvm_context.patch (2.26 KB)

What're the situation in which you need to clear it? If there are
outstanding users of it (given that you mention clients possibly
holding references to the pimpl, it sounds like you might have
outstanding users) then wouldn't they be at risk of breaking if you
mutate the LLVMContext underneath them?

& if you don't have outstanding users, is there any particular benefit
to resetting the LLVMContext compared to just making a new one?

Hi,

I'm trying to develop a way to reliably clean the LLVMContext in order to
make it possible to use it multiple times.

LLVMContext itself is an almost empty object delegating almost all its
content to LLVMContextImpl.
This makes it very clean ideally, because clearing the LLVMContext would be
as easy as deleting the LLVMContextImpl and creating a new one.

The problem is that for some reason which I'm not aware of LLVMContextImpl
is actually exposed as a public pointer in the LLVMContext interface,making
it publicly available to objects that use it directly (this seems to happen
quite a lot in the codebase).

In LLVMContext the LLVMContextImpl is contained in a pImpl pointer that is
const (the pointer itself can't be changed) and I guess this is some kind
of protection against object replacing the LLVMContextImpl directly, which
stops us from just deleting it + getting a new one.
In addition to that, being pImpl public, there is no guarantee that objects
don't rely on pImpl remaining always the same pointer.

This makes it more difficult to clear LLVMContext.

I can't comment on your solution.
But again the open source OpenGL driver stack is suffering from problems connected to
LLVMContext.

So our observation is, if we use several LLVMContexts within the program execution
lifetime, LLVM accumulates memory that is only freed on destruction of the
managed singletons. It turns out that in SDNode::getValueTypeList(EVT) the non simple
value types are cached and keyed by a llvm::Type value which in the end refers to a
LLVMContext reference value. Which in turn leads to the llvm library hogging memory
in this EVT std::set for each new LLVMContext.
For the OpenGL drivers using the global LLVMContext singleton leads to serious
threadsafeness problems. One driver is just non OpenGL conformant
non thread safe because of that, some others just tolerate LLVM hogging
memory because of that. Even worse for the ones just tolerating hogging
memory, I could potentially think of possible ABA like problems with a
new LLVMContext or llvm::Type being malloced at the same address than a previous already deleted
one but not knowing about the cached EVT's which appear to belong to this context.

That said I think you need to be careful since some of the llvm static caches refer to
LLVMContext pointers and at least the one I mentioned above is not cleared if either
a whole LLVMContext is cleared nor it is cleared by what you want to achieve.

Also for my interest:
Does anybody know of other statically cached object instances that are tied to an
LLVMContext but not cleaned up when the referring LLVMContext is destroyed?

Greetings and thanks

Mathias

Hello,

the need here is to have a single LLVMContext used for multiple compilations.

You make a good point about that by the way. If there are outstanding users cleaning the context under their seats might still pose a risk to them, and in that case deleting + newing a new LLVMContextImpl might actually not be very different.

Marcello

Hello,

the need here is to have a single LLVMContext used for multiple
compilations.

What I'm trying to understand is why is that a need - is that not
equivalent to just building a new LLVMContext for each compilation? Is
there a performance concern (with data to back it up?) about doing
that?

What is it that goes wrong if you just start the next compilation
without clearing the context in any way? (what's the persistent state
that interferes with the next run?)

- David

What you say here is pretty scary ... before saying anything about this I
think I will have a read in the source code.

If somebody in the meantime has any extra information about this it would
be useful :slight_smile:

Marcello

Right, this is likely going to have at least some of the same problems
that Bill ran into when he tried to update a TargetMachine by
placement newing a new one into it.

Can you elaborate a bit more about what you're trying to do? (I
realize this may be difficult for reasons.)

-eric

An example is that a lot of data just accumulates between different compilations, like Constants for example or MDNodes.

In the long run the memory hogged starts to become a concern.

Something like, keeping the compiler alive for a many compilation sessions, using the same LLVMContext, but without the problem of accumulating data in the LLVMContext that would slowly fill up the memory.

This as much I can enter into details that I can :confused:

Probably this is also a quite common use case scenario too.

Marcello

Right. Would make more sense to just start a new context past a
certain limit (assuming that you're done with the existing cached
information from the modules).

-eric

An example is that a lot of data just accumulates between different compilations, like Constants for example or MDNodes.

In the long run the memory hogged starts to become a concern.

It would be awesome if we could just move these to the module, but I understand that it would harm some users.

Some time ago there was a suggestion of adding some gc functionality to clear constants and metadata not used by any module.

What I'm still trying to understand is why destroying the LLVMContext
and creating a new one is insufficient for this use case. Could you
explain why that wouldn't solve your problem?

Something like, keeping the compiler alive for a many compilation sessions,
using the same LLVMContext, but without the problem of accumulating data in
the LLVMContext that would slowly fill up the memory.

That's not actually anything beyond what you've told us before.
There's certainly no reasoning or particular barriers to recreating an
LLVMContext there. Nothing to help us find either a better solution or
understand more about your situation.

This as much I can enter into details that I can :confused:

Well, fair enough. But you have to realise there'll come a point where
you're on your own if all you can say is "I need X".

Cheers.

Tim.

From: llvmdev-bounces@cs.uiuc.edu [mailto:llvmdev-bounces@cs.uiuc.edu] On Behalf Of Marcello Maggioni
Subject: Re: [LLVMdev] Making it possible to clear the LLVMContext

Something like, keeping the compiler alive for a many compilation sessions, using the same

LLVMContext, but without the problem of accumulating data in the LLVMContext that

would slowly fill up the memory.

Again, why not just delete the old Context and get a new one? That’s a whole lot simpler than trying to clean up a used one.

However, reusing an existing Context (without trying to clean it up other than removing modules from it) has some benefit, in that many of the items can be reused in subsequent compilations, since many values show up repeatedly in a given environment.

  • Chuck

Simply using a new context would certainly be easier, but idealistically we should be able to support long lived contexts.

Are the things we're "caching" truly just a cache? If so, we could pick a maximum size for the cache and evict old items. My worry is that we may rely on some objects being in the "cache" for things like pointer equality. If so, we may need a more complicated weak/strong ref management scheme.

Philip

I can't comment on your solution.
But again the open source OpenGL driver stack is suffering from problems
connected to
LLVMContext.

So our observation is, if we use several LLVMContexts within the program
execution
lifetime, LLVM accumulates memory that is only freed on destruction of the
managed singletons. It turns out that in SDNode::getValueTypeList(EVT) the
non simple
value types are cached and keyed by a llvm::Type value which in the end
refers to a
LLVMContext reference value. Which in turn leads to the llvm library
hogging memory
in this EVT std::set for each new LLVMContext.
For the OpenGL drivers using the global LLVMContext singleton leads to
serious
threadsafeness problems. One driver is just non OpenGL conformant
non thread safe because of that, some others just tolerate LLVM hogging
memory because of that. Even worse for the ones just tolerating hogging
memory, I could potentially think of possible ABA like problems with a
new LLVMContext or llvm::Type being malloced at the same address than a
previous already deleted
one but not knowing about the cached EVT's which appear to belong to this
context.

It looks like this bug is in SDNode::getValueTypeList(). This is a real
bug that should be filed and fixed.

That said I think you need to be careful since some of the llvm static

caches refer to
LLVMContext pointers and at least the one I mentioned above is not cleared
if either
a whole LLVMContext is cleared nor it is cleared by what you want to
achieve.

Also for my interest:
Does anybody know of other statically cached object instances that are
tied to an
LLVMContext but not cleaned up when the referring LLVMContext is destroyed?

Not that I know of. Any global that points to a Type* is a potential bug
and should probably be removed.

I can't comment on your solution.
But again the open source OpenGL driver stack is suffering from problems
connected to
LLVMContext.

So our observation is, if we use several LLVMContexts within the program
execution
lifetime, LLVM accumulates memory that is only freed on destruction of the
managed singletons. It turns out that in SDNode::getValueTypeList(EVT)
the non simple
value types are cached and keyed by a llvm::Type value which in the end
refers to a
LLVMContext reference value. Which in turn leads to the llvm library
hogging memory
in this EVT std::set for each new LLVMContext.
For the OpenGL drivers using the global LLVMContext singleton leads to
serious
threadsafeness problems. One driver is just non OpenGL conformant
non thread safe because of that, some others just tolerate LLVM hogging
memory because of that. Even worse for the ones just tolerating hogging
memory, I could potentially think of possible ABA like problems with a
new LLVMContext or llvm::Type being malloced at the same address than a
previous already deleted
one but not knowing about the cached EVT's which appear to belong to this
context.

It looks like this bug is in SDNode::getValueTypeList(). This is a real
bug that should be filed and fixed.

Doing some archaeology, it seems like the blame on that cache points at
r78610. The commit message is "Rename MVT to EVT, [...]", so it seems like
it goes back even further. Probably just a remnant of something ancient.
Anyone that was around back then have any insight into this?

-- Sean Silva

Hi,

I'm trying to develop a way to reliably clean the LLVMContext in order to
make it possible to use it multiple times.

LLVMContext itself is an almost empty object delegating almost all its
content to LLVMContextImpl.
This makes it very clean ideally, because clearing the LLVMContext would be
as easy as deleting the LLVMContextImpl and creating a new one.

The problem is that for some reason which I'm not aware of LLVMContextImpl
is actually exposed as a public pointer in the LLVMContext interface,making
it publicly available to objects that use it directly (this seems to happen
quite a lot in the codebase).

In LLVMContext the LLVMContextImpl is contained in a pImpl pointer that is
const (the pointer itself can't be changed) and I guess this is some kind
of protection against object replacing the LLVMContextImpl directly, which
stops us from just deleting it + getting a new one.
In addition to that, being pImpl public, there is no guarantee that objects
don't rely on pImpl remaining always the same pointer.

This makes it more difficult to clear LLVMContext.

I can't comment on your solution.
But again the open source OpenGL driver stack is suffering from problems connected to
LLVMContext.

So our observation is, if we use several LLVMContexts within the program execution
lifetime, LLVM accumulates memory that is only freed on destruction of the
managed singletons. It turns out that in SDNode::getValueTypeList(EVT) the non simple
value types are cached and keyed by a llvm::Type value which in the end refers to a
LLVMContext reference value. Which in turn leads to the llvm library hogging memory
in this EVT std::set for each new LLVMContext.
For the OpenGL drivers using the global LLVMContext singleton leads to serious
threadsafeness problems. One driver is just non OpenGL conformant
non thread safe because of that, some others just tolerate LLVM hogging
memory because of that. Even worse for the ones just tolerating hogging
memory, I could potentially think of possible ABA like problems with a
new LLVMContext or llvm::Type being malloced at the same address than a previous already deleted
one but not knowing about the cached EVT's which appear to belong to this context.

That said I think you need to be careful since some of the llvm static caches refer to
LLVMContext pointers and at least the one I mentioned above is not cleared if either
a whole LLVMContext is cleared nor it is cleared by what you want to achieve.

Also for my interest:
Does anybody know of other statically cached object instances that are tied to an
LLVMContext but not cleaned up when the referring LLVMContext is destroyed?

This is probably an independent bug, no? How about moving the evt cache to the llvmcontext so that it goes away with it?

Hi,

This is probably an independent bug, no? How about moving the evt cache to the llvmcontext so that it goes away with it?

Yes this is independent in that sense. But this
illustrates that the guy with the original message has to be careful if he is trying to do
what he wanted - so this is connected as such. May be as a counter example why he
should expect may be subtle problems when he attempts to clear a
LLVMContext at the first place.

I have not looked too deep into llvm to see if your suggestion would be possible or not.
The only naive approach that I quickly checked at that time was that if the context would prune its
own cache entries we would introduce a circular library dependency.
It would be great to have a solution for this.
It's sad that I cannot promise to look into that, I have already done this too often at
different projects creating a too long backlog on my side to be useful with this. Sorry ...

Greetings

Mathias

Hi,