Strange LLVM Crash

I'm implementing a JIT and getting some strange crashes. I'm unsure exactly
what's causing them, but it seems to occur when I call the getReturnType()
method on some LLVM function objects. More precisely, I'm registering some
native C++ functions as LLVM functions through the addGlobalMapping method
of an execution engine object. I then keep a pointer to those LLVM function
objects, which I use later on to create call instructions to those native
C++ functions.

    // Create a function type object for the function
    llvm::FunctionType* pFuncType = llvm::FunctionType::get(returnType,
argTypes, false);
    
    // Create a function object with external linkage and the specified
function type
    llvm::Function* pFuncObj = llvm::Function::Create(
      pFuncType,
      llvm::Function::ExternalLinkage,
      name,
      s_pModule
    );
    
    // Add a global mapping in the execution engine for the function pointer
    s_pExecEngine->addGlobalMapping(pFuncObj, pFuncPtr);

The crash does not occur the first time I JIT compile a function, but the
second or third time I compile some other function, I will get a crash when
trying to create a call instruction. If I try to call the getReturnType()
method on the function object before creating the call, I also get a crash.
All of my functions are getting put into the same LLVM module, so I'm
wondering if LLVM somehow invalidates my function object pointers when I JIT
functions, perhaps when I run optimization passes on my LLVM module? Should
I not keep pointers to those function objects?

Any hints will be appreciated.

Hi Nyx,

I'm implementing a JIT and getting some strange crashes. I'm unsure exactly
what's causing them, but it seems to occur when I call the getReturnType()
method on some LLVM function objects. More precisely, I'm registering some
native C++ functions as LLVM functions through the addGlobalMapping method
of an execution engine object. I then keep a pointer to those LLVM function
objects, which I use later on to create call instructions to those native
C++ functions.

sounds like memory corruption to me. Try running under valgrind.

Ciao,

Duncan.

Valgrind seems to not like the Boehm's GC and cause a seg fault of its own
before I can even test anything.

Anyways, I don't see how I could be causing "memory corruption". If I do
something wrong, it has to be some kind of unintended use of the LLVM API.
I'm definitely not writing data into the LLVM function objects myself. Could
LLVM be deleting those objects is the question I'm basically asking.

- Maxime

Duncan Sands wrote:

Ok, well, I seem to have pinpointed the cause of the problem more accurately.
I'm running some optimization passes on my module after I compile each
function in my scripting language (functions can be compiled at various
times, when scripts are loaded). Now it seems these optimization passes will
prune some of the native C++ functions I'm registering in my module (the
functions that haven't been called/used yet). I'd like to find a way to
disable this so that all the native functions I register will stay in the
module.

Here are the optimization passes I'm running:

  passManager.add(new llvm::TargetData(s_pModule));
  passManager.add(llvm::createLowerSetJmpPass());
  passManager.add(llvm::createRaiseAllocationsPass());
  passManager.add(llvm::createCFGSimplificationPass());
  passManager.add(llvm::createPromoteMemoryToRegisterPass());
  passManager.add(llvm::createGlobalOptimizerPass());
  passManager.add(llvm::createGlobalDCEPass());
  passManager.add(llvm::createFunctionInliningPass());

I would like to know either which pass does this (global optimizer maybe?)
so I can disable it, or what flag I can set on my C++ function objects to
keep them from being pruned out.

Functions that have an entry in llvm.used should not be deleted. Not entirely sure the JIT honors that but I think it does.

Nyx wrote:

Ok, well, I seem to have pinpointed the cause of the problem more accurately.
I'm running some optimization passes on my module after I compile each
function in my scripting language (functions can be compiled at various
times, when scripts are loaded). Now it seems these optimization passes will
prune some of the native C++ functions I'm registering in my module (the
functions that haven't been called/used yet). I'd like to find a way to
disable this so that all the native functions I register will stay in the
module.

"externally visible" functions should never be deleted. What's the linkage type on your functions? internal? Don't mark things internal unless you don't mind if they go away. :slight_smile:

Nick

The linkage type is set to external, I have little code snippet I use to
register those native functions in the first post of this topic. The global
DCE pass deletes the unused native functions when run. I commented it out
for now...

Nick Lewycky wrote:

Nyx wrote:

The linkage type is set to external, I have little code snippet I use to
register those native functions in the first post of this topic. The global
DCE pass deletes the unused native functions when run. I commented it out
for now...

Can you make this happen by writing a custom .ll to demonstrate the problem? For example:

   $ cat gdce.ll
   define i32 @foo() {
     ret i32 0
   }
   $ llvm-as < gdce.ll | opt -globaldce | llvm-dis
   ; ModuleID = '<stdin>'

   define i32 @foo() {
           ret i32 0
   }

If it happens as you say, you should be able to create a .ll where "opt -globaldce" will eliminate your function.

Nick

I don't know how to do that. Rather new to LLVM. The functions that get
stripped out are native C++ functions that I'm registering in my execution
engine by doing:
  
    // Create a function type object for the function
    llvm::FunctionType* pFuncType = llvm::FunctionType::get(returnType,
argTypes, false);
    
    // Create a function object with external linkage and the specified
function type
    llvm::Function* pFuncObj = llvm::Function::Create(
      pFuncType,
      llvm::Function::ExternalLinkage,
      name,
      s_pModule
    );
    
    // Add a global mapping in the execution engine for the function pointer
    s_pExecEngine->addGlobalMapping(pFuncObj, pFuncPtr);

Where pFuncPtr is a void pointer to the native C++ function. I register
several of these native functions which are meant to be runtime support for
the scripts I then JIT compile. After compiling each script, I run
optimization passes. If the global DCE one is enabled, it eliminates the
native functions that were not yet used from the module. Then, if I try to
compile other scripts that would use the functions that have been
eliminated, I get a crash.

From the LLVM source code, I found the following comment:

// createGlobalDCEPass - This transform is designed to eliminate unreachable
// internal globals (functions or global variables)

So I guess the optmization pass is doing exactly what it was meant to do. I
disabled it for now since I don't want the native functions to get
eliminated, and am currently not using any global variables.

By the way, is there any way to have the pass managers run its passes on a
single function, instead of a whole module?

- Maxime

Nick Lewycky wrote:

Nyx wrote:

I don't know how to do that. Rather new to LLVM. The functions that get
stripped out are native C++ functions that I'm registering in my execution
engine by doing:

Ah! Given that information I produce this testcase:

   $ cat gdce.ll
   declare i32 @foo()
   $ llvm-as < gdce.ll | opt -globaldce | llvm-dis
   ; ModuleID = '<stdin>'
   $

The globaldce pass eliminates declarations with no definition, even if they aren't marked internal. There's your problem.

Should this really be happening? Does global DCE offer other optimizations that JIT users would want but without deleting externally visible globals? Or should JIT users just not run global DCE?

Nick

Yes, and this is what I'd suggest since it's quite a bit faster. Basically you'd use a function pass manager there are some passes which are function only and then get pointer to function as normal.

-eric

Is there a webpage documenting these function passes? Which ones should I run
to maximize performance? Also, do the machine function passes automatically
run (I'm assuming those optimize the final x86 code).

Eric Christopher-2 wrote:

Is there a webpage documenting these function passes?

Here's some:

http://llvm.org/docs/WritingAnLLVMPass.html#FunctionPass

You can also look in llvm/lib/Transforms/Scalar for runOnFunction ()

Which ones should I run
to maximize performance?

There's no right way to determine this. It depends on what you need/want from your code.

Also, do the machine function passes automatically
run (I'm assuming those optimize the final x86 code).

Yes. Don't forget to add the target data pass.

-eric

This is what I have so far:

  // Add optimization passes to the function passes
  s_pFunctionPasses->add(new llvm::TargetData(s_pModule));
  s_pFunctionPasses->add(llvm::createCFGSimplificationPass());
  s_pFunctionPasses->add(llvm::createPromoteMemoryToRegisterPass());
  s_pFunctionPasses->add(llvm::createConstantPropagationPass());
  s_pFunctionPasses->add(llvm::createDeadCodeEliminationPass());
  s_pFunctionPasses->add(llvm::createInstructionCombiningPass());

Where s_pFunctionPasses is a FunctionPassManager I create when initializing
my program and delete at shutdown. I'm running the CFG simplification one
first in the hope that it will speed up the passes that follow it.

Just to make sure I understand: I shouldn't add x86 optimization passes
myself, LLVM will take care of running the appropriate ones for me, is that
correct? If it's incorrect, how do I manage those passes?

- Maxime

Eric Christopher-2 wrote:

This is what I have so far:

  // Add optimization passes to the function passes
  s_pFunctionPasses->add(new llvm::TargetData(s_pModule));
  s_pFunctionPasses->add(llvm::createCFGSimplificationPass());
  s_pFunctionPasses->add(llvm::createPromoteMemoryToRegisterPass());
  s_pFunctionPasses->add(llvm::createConstantPropagationPass());
  s_pFunctionPasses->add(llvm::createDeadCodeEliminationPass());
  s_pFunctionPasses->add(llvm::createInstructionCombiningPass());

Where s_pFunctionPasses is a FunctionPassManager I create when initializing
my program and delete at shutdown. I'm running the CFG simplification one
first in the hope that it will speed up the passes that follow it.

I'd swap it and mem2reg, but otherwise it looks ok. You can also add GVN if you can spare the time.

Just to make sure I understand: I shouldn't add x86 optimization passes
myself, LLVM will take care of running the appropriate ones for me, is that
correct? If it's incorrect, how do I manage those passes?

Unless you have some x86 specific machine code pass that you wrote and need to run, then no, you don't need to worry about it. getPointerToFunction() should handle all of that.

-eric