using Clang for runtime compilation of a class

Dear Clang Users,

I have a particular programming problem, and several discussion forums suggested Clang as a possible solution. Being totally new to Clang and its community, is this the right place to ask questions about how to use Clang for a particular programming task? If not, where should I turn to?

In case this is the right forum for such questions, I continue describing my problem. Basically, I want to compile a class which inherits from a virtual interface class at runtime of my program. The program should then create an instance of this new class and use it through its interface.

Let me quickly sketch an example. The static version of my program would look like this:

class Base
{
   public:
     virtual int calc() const;
};

class Derived: public Base
{
   public:
     virtual int calc() const
     {
       return 42;
     }
};

int main()
{
     std::cout << Derived().calc() << std::endl;
     return 0;
}

The dynamic version (with two fuzzy lines of code) shall look like this:

class Base
{
   public:
     virtual void calc() const;
};

int main()
{
   std::string code = read_derived_code();

   // two fuzzy lines for which I am looking for a solution
   SomeCompiledObjectClass o = compile(code);
   Base* d = SomeFactoryClass(o).createInstance();

   std::cout << d->calc() << std::endl;
   return 0;
}

Is Clang the right tool for solving my two fuzzy lines of code? If yes, which class should have a look at? If not, do you maybe know about another library/compile which can achieve this task?

Thanks a lot for your help!

Cheers,
Georg.

Dear Clang Users,

I have a particular programming problem, and several discussion forums
suggested Clang as a possible solution. Being totally new to Clang and its
community, is this the right place to ask questions about how to use Clang
for a particular programming task? If not, where should I turn to?

In case this is the right forum for such questions, I continue describing
my problem. Basically, I want to compile a class which inherits from a
virtual interface class at runtime of my program. The program should then
create an instance of this new class and use it through its interface.

Let me quickly sketch an example. The static version of my program would
look like this:

class Base
{
  public:
    virtual int calc() const;
};

class Derived: public Base
{
  public:
    virtual int calc() const
    {
      return 42;
    }
};

int main()
{
    std::cout << Derived().calc() << std::endl;
    return 0;
}

The dynamic version (with two fuzzy lines of code) shall look like this:

class Base
{
  public:
    virtual void calc() const;
};

int main()
{
  std::string code = read_derived_code();

  // two fuzzy lines for which I am looking for a solution
  SomeCompiledObjectClass o = compile(code);
  Base* d = SomeFactoryClass(o).createInstance();

  std::cout << d->calc() << std::endl;
  return 0;
}

Is Clang the right tool for solving my two fuzzy lines of code? If yes,
which class should have a look at? If not, do you maybe know about another
library/compile which can achieve this task?

This migth be the sort of thing that would escalate to cfe-dev and/or
llvm-dev.

What you're looking to do is JIT (Just In Time) compile some code. That
involves using clang to turn the source code into LLVM IR and then LLVM to
make machine code from that, put it in memory, make it executable, and call
some part of it.

Your two fuzzy lines of code would look slightly different (chances are
what you'd do is generate source code that included a private function that
return an instance, then you would JIT compile that function, indirectly
code generating anything in the class that was needed, etc).

I'm not sure where the best examples of the Clang half of this problem are.
But for the second have, LLVM has the Kaleidoscope tutorials (especially
the recently added ORC examples) that show how to use LLVM as a JIT
compiler. You shuold be able to wrap up all these APIs into a fairly simple
API for your one specific use case.

I think /maybe/ out there is an existing C++ rapid development environment
using Clang (essentially a C++ interpreter/interactive command prompt like
Python) that you might be able to grab the basics of "feed code to
compiler, get IR" from.

- David

Hi Georg (and others interested in Clang+JIT),

We just implemented this (Clang+JIT) in our open-source project CasADi (http://casadi.org). Feel free to have a look and/or reuse components.

We used a plugin-design for using Clang, which is a bit different from the examples we could find for using LLVM/Clang. We added a generic “Compiler” class and made Clang/JIT a plugin for this Compiler class. Here is the implementation of the plugin: https://github.com/casadi/casadi/blob/develop/casadi/interfaces/clang/clang_compiler.cpp. We also made a second plugin (only for Linux/Mac) which simply calls the system compiler using shell commands. Mainly for benchmarking and when Clang/JIT isn’t available: https://github.com/casadi/casadi/blob/develop/casadi/solvers/shell_compiler.cpp.

The trickiest was to set up the build system (we use CMake) and in particular redistribute the C and C++ standard libraries needed to compile any useful C/C++ code. And do it in a way that makes the compiled code portable and not relying on e.g. setting environmental variables.

This all works now, also on Windows, although it is not yet completely mature (especially when trying to JIT compile C++ code). The solution used in the LLVM examples was to use the arguments passed to “main”, e.g. to locate the “resource”, but this didn’t work for us since or tool is itself a library and we don’t have a “main”. So the design we chose instead was to place the compilation resources (including C/C++ standard library headers) in a relative location to the plugin (which is a shared library) since the path of the plugin can be retrieved at runtime by “dladdr” in Linux/Mac and GetModuleHandleEx+GetModuleFileName on Windows.

Best regards,
Joel

Hi Georg, Joel,

Georg - Your question got me curious, so I wrote up a very basic implementation based on the clang-interpreter demo from clang/examples. I’ve tested it on Darwin with LLVM/Clang r249281 and it works, in its very limited way. I thought I would share it, in case it helps.

It sounds like Joel’s project will be an excellent resource for a more serious approach.

Cheers,
Lang.

jit-derived.tgz (3.74 KB)