LLVM for JIT only use

Project Infrastructure LLVM Dev List Archives
1

Dear All,

I downloaded the latest CVS of llvm

I want to try to use LLVM only for JIT only use (generating code in
memory for x86 and, when available PowerPC [32bits])

How do I ./configure to avoid being asked about llvmgcc, which, in my
very partial understanding, is only used to parse C code... If I
understand correctly what Chris Lattner emailed me, LLVM uses only GCC
frontend for C & C++ parsing. I don't want to parse C or C++.

I would just want to build the necessary libraries to compile and run
llvm/examples/HowToUseJIT/HowToUseJIT.cpp and other similar examples.

I want to use LLVM directly from its C++ API to build the "bytecode"
tree (actually the sequence or tree of LLVM instructions), so I really
don't need any C or C++ parser.

(FWIW, I did know and use GNU lightning)

Regards.

2

I want to try to use LLVM only for JIT only use (generating code in
memory for x86 and, when available PowerPC [32bits])

Cool! FYI, the PowerPC JIT needs work to make it operational, but the
x86 JIT is good to go.

How do I ./configure to avoid being asked about llvmgcc, which, in my
very partial understanding, is only used to parse C code... If I
understand correctly what Chris Lattner emailed me, LLVM uses only GCC
frontend for C & C++ parsing. I don't want to parse C or C++.

Don't worry about the configure errors at the end, and just run
"make tools-only". That will build all the libraries and LLVM tools,
but not build the runtime libraries (which require llvm-gcc).

I would just want to build the necessary libraries to compile and run
llvm/examples/HowToUseJIT/HowToUseJIT.cpp and other similar examples.

They will be built automatically at the end of the build.

If you really don't want to link any of the tools, comment them out in
tools/Makefile (set PARALLEL_DIRS to nothing). You can't just go into
llvm/lib and run 'make' because some of the libraries need to have
utilities built first (llvm/utils) which, in turn, need some basic
libraries (Support and System).

I want to use LLVM directly from its C++ API to build the "bytecode"
tree (actually the sequence or tree of LLVM instructions), so I really
don't need any C or C++ parser.

Makes sense.

HTH,

3

Hello List,

(apparently this list is moderated)

Le/On Sun, Nov 07, 2004 at 03:45:41PM -0600, Misha Brukman
écrivait/wrote:

I want to try to use LLVM only for JIT only use (generating code in
memory for x86 and, when available PowerPC [32bits])

If you want to know I am experimenting some JIT generation for a
experimental toy functional language [this is only a hobby] and I did
write the OcamlJIT (a JIT translator of Ocaml bytecode using GNU
lightning)

Cool! FYI, the PowerPC JIT needs work to make it operational, but the
x86 JIT is good to go.

Don't worry about the configure errors at the end, and just run
"make tools-only". That will build all the libraries and LLVM tools,
but not build the runtime libraries (which require llvm-gcc).

First, I am extremely confused by the terminology. IMHO, the runtime
libraries would include (from my JIT only perspective)

1. all the LLVM libraries required to JIT-generate machine code, and
to build a representation of the LLVM instruction sequence or tree.

Z. the garbage collection I'm coding for my toy language.

So I really can't figure why is the LVM-GCC parser needed here!

For me the runtime stuff of LLVM is the stuff I need to JIT (without C
parsing) - but I am wrong! what exactluy is the "runtime" and why does
it require llvm-gcc (I thought that LLVM is not a bootstrapped
language: the compiler is not written in LLVM...)

Second trivial question: only I "make tools-only" how can I install
the stuff (in the standard /usr/local/ prefix)? Does "sudo make
install" is enough?

Regards.

4

(apparently this list is moderated)

Yes, we get too much spam, so it's moderated, but only with regard to
spam, nothing else.

>> I want to try to use LLVM only for JIT only use (generating
code in Basile>> memory for x86 and, when available PowerPC [32bits])

If you want to know I am experimenting some JIT generation for a
experimental toy functional language [this is only a hobby] and I did
write the OcamlJIT (a JIT translator of Ocaml bytecode using GNU
lightning)

Sounds pretty neat.

First, I am extremely confused by the terminology. IMHO, the runtime
libraries would include (from my JIT only perspective)

1. all the LLVM libraries required to JIT-generate machine code, and
to build a representation of the LLVM instruction sequence or tree.

Z. the garbage collection I'm coding for my toy language.

So I really can't figure why is the LVM-GCC parser needed here!

Sorry for the confusion. By 'runtime', I mean 'llvm/runtime' directory,
which are C libraries for our purposes (profiling, tracing, a beginning
of a garbage-collection library, zlib, etc.). Take a look inside -- you
can tell what things are based on their names.

The one important thing that the runtime/ directory provies is the
crtend.o that is linked to any C/C++ program that LLVM-GCC compiles, but
again, that is of no relevance to your project.

For me the runtime stuff of LLVM is the stuff I need to JIT (without C
parsing) - but I am wrong! what exactluy is the "runtime" and why does
it require llvm-gcc (I thought that LLVM is not a bootstrapped
language: the compiler is not written in LLVM...)

In your list, (1) is llvm/lib/VMCore and a few others, (2) you provide.
We just have a different terminology.

Bottom line: you don't need llvm/runtime.

Second trivial question: only I "make tools-only" how can I install
the stuff (in the standard /usr/local/ prefix)? Does "sudo make
install" is enough?

That should do it, but to be certain, I would defer that question to one
of our Makefile gurus Reid Spencer.

5

Le Sun, Nov 07, 2004 at 04:14:19PM -0600, Misha Brukman écrivait/wrote:

First, I am extremely confused by the terminology. IMHO, the runtime
libraries would include (from my JIT only perspective)

1. all the LLVM libraries required to JIT-generate machine code, and
to build a representation of the LLVM instruction sequence or tree.

Z. the garbage collection I'm coding for my toy language.

So I really can't figure why is the LVM-GCC parser needed here!

Sorry for the confusion. By 'runtime', I mean 'llvm/runtime' directory,

In your list, (1) is llvm/lib/VMCore and a few others, (2) you provide.
We just have a different terminology

I did "make" inside llvm/examples/ and inside
llvm/examples/HowToUseJIT/ and expected to get an ELF executable.. But
I've only got the 2 Release/HowToUseJIT.[do] files..

But the llvm/examples/HowToUseJIT/HowToUseJIT.cpp has at line 47 a
"int main()" function. So I was waiting for an executable which I
would be able to run. (FYI, the GNU lightning library provide a tests/
directory which can easily compile a few test stuff, which build plain
executables showing simple JIT generation&execution within the same
process, and so does libjit).

Bottom line: you don't need llvm/runtime.

What are the exact libraries I'll need from LLVM (to JIT generate
machine code)? Where are these (I would like the list of all the
lib*.{a,so} and all headers <*.h> in /usr/local/ on my
Linux/Debian/Sid/x86 system)? How are these supposed to be installed
(if possible, I want to avoid llvm-gcc, because I hpave trouble
compiling the latest CVS of it, and because I believe I really don't
need it)?

How can I tell LLVM the address where I want the generated code to go
(or how can I get this address, preferably with a "destructor"
function to release the memory)? I coded a garbage collector which
should garbege generated code...

How do I plug my copying GC into LLVM? It has to know about the stack
frames, etc... (I mostly need the location inside the stack frames of
GC-ed pointers)

I'm sorry to ask such basic questions, but the documentation don't
help me much (and I am a bit lost in it.. I don't care about any kind
of llvm-gcc or llvm-ld, I just want to generate and then run code in
memory!)

If some kind LLVM person is in charge of LLVM documentation, I am
missing a "using LLVM for JIT machine-code generation for dummies"
page -which would explain the examples/HowToUseJIT/ and similar-
Apparently, JIT is a by-product of LLVM, but it is the main feature
that attracts me here...

BTW, are there some European LLVM users & developers (working & living
in Europe) or is LLVM only a US software?

Again, sorry for asking such basic questions... I'm a bit lost in the
(intimidating) documentation....

Regards.

6

I did "make" inside llvm/examples/ and inside
llvm/examples/HowToUseJIT/ and expected to get an ELF executable.. But
I've only got the 2 Release/HowToUseJIT.[do] files..

All executables are placed in tools/$(BUILD_TYPE), in your case,
llvm/tools/Release. That is there from the time before we had the
examples/ directory, and the only executables produced were the tools.

Perhaps that should be changed to build into examples/$(BUILD_TYPE) ...

> Bottom line: you don't need llvm/runtime.

What are the exact libraries I'll need from LLVM (to JIT generate
machine code)?

Look through llvm/tools/Makefile.JIT -- it adds JIT support libraries to
the USEDLIBS variable, so for x86, for example:

LLVMJIT LLVMCodeGen LLVMExecutionEngine LLVMX86 LLVMSelectionDAG
LLVMInterpreter LLVMScalarOpts LLVMAnalysis.a LLVMTransformUtils.a
LLVMBCReader LLVMCore LLVMSupport.a LLVMTarget.a LLVMSystem.a

For the ones without an extension, append .o, for the ones with .a,
prepend 'lib' and look in llvm/lib/Release .

Where are these (I would like the list of all the lib*.{a,so} and all
headers <*.h> in /usr/local/ on my Linux/Debian/Sid/x86 system)?

The headers are llvm/include/llvm/*, however, I can't tell you
immediately what is the _minimal_ set of headers that will make a JIT
application compile, because your JIT may use any of the LLVM
functionality (theoretically). You could see what any of the examples
#includes (HowToUseJIT, Fibonacci, etc) but some headers #include
others...

How are these supposed to be installed (if possible, I want to avoid
llvm-gcc, because I hpave trouble compiling the latest CVS of it, and
because I believe I really don't need it)?

You don't need llvm-gcc for these libraries or headers. Theoretically,
you would install these in something like /usr/local/include/llvm/* and
/usr/local/lib/* or something similar to that. 'make install' should
install into $prefix/ include, lib, etc.

How can I tell LLVM the address where I want the generated code to go
(or how can I get this address, preferably with a "destructor"
function to release the memory)? I coded a garbage collector which
should garbege generated code...

At present, you cannot tell the JIT *where* you want the code to go.
Machine code destruction is currently not implemented (but you may
find it implemented in the near future). In the mean time, you can use
this function to tell the ExecutionEngine to release memory for
generated code:

  ExecutionEngine::freeMachineCodeForFunction(Function *F);

Note that you'll have to update your sources to get the changes from CVS
for this.

How do I plug my copying GC into LLVM? It has to know about the stack
frames, etc... (I mostly need the location inside the stack frames of
GC-ed pointers)

I'm not the person to ask about GC, I'll let Chris Lattner chime in on
this. You can read a previous discussion about scheme GC and LLVM:

  http://mail.cs.uiuc.edu/pipermail/llvmdev/2004-July/thread.html#1524

Or you can also read up on the scheme compiler directly:

  http://llvm.cs.uiuc.edu/ProjectsWithLLVM/#scheme

I'm sorry to ask such basic questions, but the documentation don't
help me much (and I am a bit lost in it.. I don't care about any kind
of llvm-gcc or llvm-ld, I just want to generate and then run code in
memory!)

If some kind LLVM person is in charge of LLVM documentation, I am
missing a "using LLVM for JIT machine-code generation for dummies"
page -which would explain the examples/HowToUseJIT/ and similar-
Apparently, JIT is a by-product of LLVM, but it is the main feature
that attracts me here...

Thanks for the feedback! We'll see about including such JIT-specific
"howto" guide to LLVM document library.

BTW, are there some European LLVM users & developers (working & living
in Europe) or is LLVM only a US software?

LLVM is used world-wide. Specifically in Europe, I believe there are
LLVM users/developers in (at least) Denmark, England, France, Germany,
Italy, Norway, Russia, Spain, and Sweden. Sorry if I'm missing anyone,
this is what I remember off the top of my head.

We don't keep tabs on them, but many of them do read this list so you'll
hear from them, don't worry. :slight_smile:

7

This is documented fairly extensively here:
http://llvm.org/docs/GarbageCollection.html#gcimpl

-Chris

8

I'm in a group working on garbage collector too and thought
GarbageCollection.html didn't answer all my questionss about finding
pointers. Chris gave some ideas for extensions to the gc_ interface in
http://mail.cs.uiuc.edu/pipermail/llvmdev/2004-October/002503.html
which may apply to your work.

Tom