Understanding BuryPointer and leak detection

Hi all,

I’m experimenting with libclang on Windows as a mechanism for runtime compilation of C code. So far results are encouraging, but I’m encountering a lot of rogue memory usage the longer the host app runs (and the more Clang code I call). Right now I’m relying on the Visual C++ CRT leak check functionality to monitor memory leakage; it’s crude and simplistic but highly reliable once you compensate for a couple of quirks. I should state up-front that I can confirm my results of leakage using other tools as well.

The CRT leak check is finding several hundred KB of leaks each time I compile a module using the embedded Clang. Strategic placement of memory breakpoints suggests that the bulk of these are due to things that are (directly or otherwise) owned by something that eventually winds up handed off to the BuryPointer() function.

I’ve skimmed the list history for the origins of BuryPointer, but I have to confess I’m still rather confused. From an outsider’s perspective, it seems like BuryPointer is just a way to work around having to actually clean up resources that have nontrivial ownership semantics. I gather that the purpose is to make the memory look like it isn’t leaked by ensuring that the pointers are still reachable. However, this doesn’t really change the fact that the memory is essentially no longer owned by anything (semantically) and is, at least conceptually, a genuine leak.

Given that my experiments show memory usage increasing linearly with the number of times I invoke libclang to do a compilation, I’m guessing that this is actually a genuine problem and that BuryPointer() is papering over the symptoms for certain leak checking tools.

So, three questions:

  • Is my understanding of BuryPointer() correct, or is there some complexity I’m not seeing as an (admittedly naïve) outsider?

  • Given that this represents a very real upper bound on how many times my host app can do compilations, is there any recourse for reclaiming this memory in some fashion?

  • Would there be interest in me contributing patches to address these leaks (and others I’ve found in both Clang and LLVM), presuming that there is agreement that it’s a problem?

Thanks,

  • Mike

My understanding is that all uses of BuryPointer should be somehow guarded by getFrontendOpts().DisableFree (or CodeGenOpt.DisableFree); can you see if DisableFree is somehow being set? Check FrontendAction::EndSourceFile.

There is also a FIXME for some “resetAndLeak*” calls not guarded by DisableFree in FrontendAction::EndSourceFile, but unless you’re doing something with AST files from libclang (is that possible?) those shouldn’t be an issue.

Ha, good catch. Turns out -disable-free is one of the command arguments generated by the driver code somewhere in the clang::driver::Driver::BuildCompilation() pipe… looks like Clang::ConstructJob().

This seems problematic for anyone using the library for multiple compilations (as is my situation) due to the implicit leakage. From the code it looks like this behavior goes away if Compilation::isForDiagnostics() returns true, but I have no idea what the other side effects of that check might be.

Any advice would be appreciated!

  • Mike

Manuel, have you guys run into this sort of leak due to defaulting to -disable-free before? It certainly seems like really unexpected behavior for clang as a library.

– Sean Silva

Manuel, have you guys run into this sort of leak due to defaulting to -disable-free before? It certainly seems like really unexpected behavior for clang as a library.

Yep. We’re not using libclang though and the libtooling entry points enable all cleanup.