Unexpected memory occupy when load bitcode module

Hello,
With llvm-11, we use the api to construct JIT (postgresql), but the memory allocation seems confusing.
With the jit running, our heap memory continue growing, but the module we will release after every sql finished, the memory on this situation we’re running shouldn’t continue increase.

The allocation stack is like this (catched by valgrind with massif tool):

 27089  65 42,357,374,233,810    1,015,789,544      969,709,287    46,080,257            0
│ 27090 95.46% (969,709,287B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
│ 27091 ->78.85% (800,970,858B) 0x12ACC449: llvm::allocate_buffer(unsigned long, unsigned long) (MemAlloc.cpp:15)
│ 27092 | ->56.96% (578,624,602B) 0x1038013A: llvm::MallocAllocator::Allocate(unsigned long, unsigned long) (AllocatorBase.h:85)
│ 27093 | | ->52.81% (536,481,792B) 0x104F8F85: llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::StartNewSlab() (Allocator.h:335)
│ 27094 | | | ->52.81% (536,481,792B) 0x104EF480: llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::Allocate(unsigned long, llvm::Align) (Allocator.h:188)
│ 27095 | | |   ->52.81% (536,481,792B) 0x104E6398: llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::Allocate(unsigned long, unsigned long) (Allocator.h:202)
│ 27096 | | |     ->26.00% (264,110,080B) 0x10559065: void* operator new<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>(unsigned long, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096u
│ 27097 | | |     | ->16.52% (167,763,968B) 0x12992FE0: llvm::PointerType::get(llvm::Type*, unsigned int) (Type.cpp:670)
│ 27098 | | |     | | ->15.66% (159,100,928B) 0x126CD4F1: (anonymous namespace)::BitcodeReader::parseTypeTableBody() (BitcodeReader.cpp:1776)
│ 27099 | | |     | | | ->15.66% (159,100,928B) 0x126CCD02: (anonymous namespace)::BitcodeReader::parseTypeTable() (BitcodeReader.cpp:1670)
│ 27100 | | |     | | |   ->15.66% (159,100,928B) 0x126D8D7E: (anonymous namespace)::BitcodeReader::parseModule(unsigned long, bool, llvm::function_ref<llvm::Optional<std::__cxx11::basic_st
│ 27101 | | |     | | |     ->15.66% (159,100,928B) 0x126DA336: (anonymous namespace)::BitcodeReader::parseBitcodeInto(llvm::Module*, bool, bool, llvm::function_ref<llvm::Optional<std::__cx
│ 27102 | | |     | | |       ->15.66% (159,100,928B) 0x126ECF78: llvm::BitcodeModule::getModuleImpl(llvm::LLVMContext&, bool, bool, bool, llvm::function_ref<llvm::Optional<std::__cxx11::ba
│ 27103 | | |     | | |         ->15.66% (159,092,736B) 0x126ED26A: llvm::BitcodeModule::getLazyModule(llvm::LLVMContext&, bool, bool) (BitcodeReader.cpp:6623)
│ 27104 | | |     | | |         | ->15.66% (159,092,736B) 0x126EE38D: llvm::getLazyBitcodeModule(llvm::MemoryBufferRef, llvm::LLVMContext&, bool, bool) (BitcodeReader.cpp:6775)
│ 27105 | | |     | | |         |   ->15.66% (159,092,736B) 0x126EE432: llvm::getOwningLazyBitcodeModule(std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >&&, llv
│ 27106 | | |     | | |         |     ->15.66% (159,092,736B) 0x126C6639: LLVMGetBitcodeModuleInContext2 (BitReader.cpp:112)

So what kind of problem we have met in this situation? Thanks for any suggestion?

Some parts of LLVM IR is loaded into the LLVMContext, rather than the llvm::Module specifically - if you keep loading IR into the same LLVMContext you will see some amount of growing memory usage.

To avoid this, dispose of the LLVMContext and make a new one from time to time.

(@lhames for JIT things)

Some parts of LLVM IR is loaded into the LLVMContext, rather than the llvm::Module specifically - if you keep loading IR into the same LLVMContext you will see some amount of growing memory usage.

Yeah – I think that’s probably what is happening here.

I’d recommend trying one LLVMContext per Module.

Thanks for all of your reply!

Besides, if we use the api LLVMOrcCreateNewThreadSafeModule to change our module to a thread safe module, and use the LLVMOrcLLJITAddLLVMIRModuleWithRT to take over this TSM, should we need to take care of the TSM’s memory resources?

No: LLVMOrcCreateNewThreadSafeModule returns an LLVMOrcThreadSafeModuleRef that owns the resources, and LLVMOrcLLJITAddLLVMIRModuleWithRT transfers this ownership to the JIT. The module and context lifetimes will be managed by the JIT from that point on.

1 Like

Appreciate for your reply , there are two small questions.

  1. If i want to release the module’s memory resources which held by JIT, i need dispose the whole JIT ?
  2. I saw LLVMOrcLLJITAddLLVMIRModuleWithRT has a resource tracker LLVMOrcResourceTrackerRef args, does this struct impact the resources management with JIT?
  1. If i want to release the module’s memory resources which held by JIT, i need dispose the whole JIT?

No. The JIT will hold the module only as long as it is needed: If you trigger compilation of the module by looking up one of its symbols then the ThreadSafeModule will be thrown away as soon as compilation is done. Or you can remove the resource tracker – see below.

  1. I saw LLVMOrcLLJITAddLLVMIRModuleWithRT has a resource tracker LLVMOrcResourceTrackerRef args, does this struct impact the resources management with JIT?

Yes. ResourceTrackers track the resources associated with sets of symbols. E.g. If you add a Module that defines symbols {a, b, c} using RT, then RT is responsible for tracking whatever resources are currently held to provide the definitions of {a, b, c} (in addition to whatever else the tracker was already tracking). Initially the resource backing {a, b, c} is the module itself, and if you immediately call remove on the tracker then you will just free the module. On the other hand if you call lookup("a") and trigger compilation of the module then the module will be thrown away and the tracker now tracks the raw compiled memory for {a, b, c}.

So the right way to think about resource management is to trust that ORC will aggressively throw away unnecessary program representations, and resource trackers let you think about the parts of your JIT’d program that you want to keep or throw away, regardless of what resources happen to be backing them.

One note of caution: ResourceTrackers do not manage dependencies. If you build a program with the following call graph:

 A (aded with RT1)
   -> B (added with RT2)
     -> C (added with RT3)

You can call remove on RT2 and this will throw away all resources associated with B and leave A with a dangling reference. It’s the client’s job to track these dependencies and make sure that they don’t leave or use dangling references.

1 Like

Thank you so much! Totally understand! :+1: