Getting Started with LLVM

Project Infrastructure LLVM Dev List Archives
1

Hello! I'm considering using LLVM for several different projects in
the future, so I'm conducting a technical evaluation. So far, the
APIs and the benchmark results look absolutely great.

Things I really like:

1) LLVM is low-level enough to support most languages that run on
standard hardware.
2) The JIT examples are really straightforward.
3) The documentation is remarkably good.
4) The performance of generated code is comparable to GCC, at least on
the PowerPC.
5) The LLVM developers appear sufficiently ambitious to encourage
real-world use of LLVM. :slight_smile:

After reading the manuals, building llvm, and messing around with
llvm-gcc--all straightforward experiences--I copied HowToUseJIT.cpp to
a new directory and tried to build it. At this point, I hit the wall
fairly hard:

1) The only supported way to link against LLVM from outside the build
directory is to use Makefile.common and the LLVM build system, both of
which require an LLVM build tree. Without extensive customization:
  a) I can't link against the installed LLVM libraries and headers.
  b) I can't use my projects' pre-existing build systems (automake,
rake, or whatever).
2) In 1.6, the "sample" project can't be built outside the LLVM tree
without modifying AutoRegen.sh to find the llvm/autoconf/m4. The
published instructions omit this step.
3) The only documentation for building code against LLVM is in Makefile.rules.

Unfortunately, it looks like integrating LLVM with an existing project
is a fair bit of work. This is certainly a problem for us, and it
might have been a problem for several other compilers that I've
contributed to in the past[1].

Ideally, it would be easy to link toy programs against LLVM (or the
LLVM JIT) without using Makefile.common. This would would let existing
projects experiment with LLVM and get a quick win or two. One approach
might be to create an 'llvm-config' script similar to the ones
provided by many Unix libraries. This might be invoked as follows:

  g++ `llvm-config --cflags` -o HowToUseJIT.o -c HowToUseJIT.cpp
  g++ `llvm-config --ldflags` -o HowToUseJIT HowToUseJIT.o `llvm-config --ldadd`

Ideally, this script would build and link against the installed copy
of LLVM, not the source and object trees left over from the build. The
script would presumably provide options for including or omitting
various LLVM components.

So here's my proposal: If you might be interested in a tool like this,
I'd be happy to work out the details with you, write the necessary
code, and send you diffs against CVS HEAD. Please let me know if such
a patch (or something similar) would be a desirable addition to LLVM.

Cheers,
Eric

[1] I've contributed code to the open source version of Gywdion Dylan,
and to Jonathan Bachrach's Goo compiler. I've also worked on a
commercial embedded compiler and several toy compilers.

2

1) The only supported way to link against LLVM from outside the build
directory is to use Makefile.common and the LLVM build system, both of
which require an LLVM build tree. Without extensive customization:
  a) I can't link against the installed LLVM libraries and headers.
  b) I can't use my projects' pre-existing build systems (automake,
rake, or whatever).

I'm currently using the "make install"-ed version of LLVM in an
autoconf/automake project. Setting it up wasn't that bad really. The .a
libraries can be statically linked with the -l flag and .o libraries are
simply linked in as normal object files (without -l). All the libraries got
installed to $(prefix)/lib and all headers went into $(prefix)/include/llvm,
just like with any other library. Also, -D__STDC_LIMIT_MACROS had to be added
to the compiler options for some header to work correctly but that was pretty
much it.

The LLVM docs have this list of libraries and their interdependencies, in case
you've missed it:
http://llvm.cs.uiuc.edu/docs/UsingLibraries.html

I asked a similar question here about a month ago and Chris answered:
"make install is lightly tested, but should work. Please report any
problems you find."

Personally, I've found LLVM to be a pleasure to work with. Its design is very
powerful, which simplifies many things a lot.

3

I'm currently using the "make install"-ed version of LLVM in an
autoconf/automake project. Setting it up wasn't that bad really. The .a
libraries can be statically linked with the -l flag and .o libraries are
simply linked in as normal object files (without -l). All the libraries got
installed to $(prefix)/lib and all headers went into $(prefix)/include/llvm,
just like with any other library. Also, -D__STDC_LIMIT_MACROS had to be added
to the compiler options for some header to work correctly but that was pretty
much it.

Thanks for your advice! I did get everything working (eventually) by pasting the following code into the Makefile for HowToUseJIT:

   .PHONY: dump-config
   dump-config:
            @echo Compile: $(Compile.CXX)
            @echo Link: $(Link)
            @echo Libs: $(LLVMUsedLibs)

...and using the result. But this is still pretty fragile, and needs be done once for every supported platform.

Gnome (and many other Unix projects with massively ugly dependencies) can be linked trivially using a "foo-config" script. If LLVM had something similar, it might save new LLVM developers several hours of digging through manuals and Makefiles.

A llvm-config script would be absolutely trivial to generate from the current Makefiles, and--if people think it would be a Good Thing--I'm volunteering to do all the legwork. :slight_smile:

I asked a similar question here about a month ago and Chris answered:
"make install is lightly tested, but should work. Please report any
problems you find."

Yup, the installed headers and libraries seem sufficient so far. Linking is slow, but that's to be expected for such a big C++ project.

Personally, I've found LLVM to be a pleasure to work with. Its design is very
powerful, which simplifies many things a lot.

Oh, you don't have to convince me of that! LLVM's design is so elegant (compared to some compiler backends I've seen) that it really ought to have theme music or something. :slight_smile:

Cheers,
Eric

4

Hi, sorry for the delay, I've been swamped lately :-/

Thanks for your advice! I did get everything working (eventually) by pasting the following code into the Makefile for HowToUseJIT:

.PHONY: dump-config
dump-config:
          @echo Compile: $(Compile.CXX)
          @echo Link: $(Link)
          @echo Libs: $(LLVMUsedLibs)

...and using the result. But this is still pretty fragile, and needs be done once for every supported platform.

Ok. This is a reasonable way to do it. FWIW, I have a similar problem in the new llvm-gcc, which links to LLVM optimizations and native targets. The makefile goop I have looks like this:

ifneq ($(LLVMBASEPATH),)
ifdef CHECKING_ENABLED
BUILDMODE=Debug
else
BUILDMODE=Release
endif
LLVMLIBPATH = $(LLVMBASEPATH)/$(BUILDMODE)/lib

# Pick the right LLVM backend based on the target triple.
LLVMTARGETOBJCMD := case $(target) in \
                      alpha-*-*) echo LLVMAlpha.o;; \
                      ia64-*-*) echo LLVMIA64.o;; \
                      i[34567]86-*-*) echo LLVMX86.o;; \
                      powerpc*-*-*) echo LLVMPowerPC.o;; \
                      sparc-*-*) echo LLVMSparc.o;; \
                      sparcv9-*-*) echo LLVMSparc.o;; \
                     esac
LLVMTARGETOBJ := $(shell $(LLVMTARGETOBJCMD))

ifeq ($(LLVMTARGETOBJ),)
$(error Unsuported LLVM Target $(target))
endif

LLVMLIBS = -L$(LLVMLIBPATH) $(LLVMLIBPATH)/$(LLVMTARGETOBJ) -lLLVMScalarOpts \
            -lLLVMTransformUtils -lLLVMAnalysis \
            $(LLVMLIBPATH)/LLVMSelectionDAG.o $(LLVMLIBPATH)/LLVMCodeGen.o \
             -lLLVMTarget $(LLVMLIBPATH)/LLVMBCWriter.o \
            $(LLVMLIBPATH)/LLVMbzip2.o \
            $(LLVMLIBPATH)/LLVMCore.o -lLLVMSupport -lLLVMSystem

uhm... *yuck*.

Gnome (and many other Unix projects with massively ugly dependencies) can be linked trivially using a "foo-config" script. If LLVM had something similar, it might save new LLVM developers several hours of digging through manuals and Makefiles.

This would be very very cool to have.

A llvm-config script would be absolutely trivial to generate from the current Makefiles, and--if people think it would be a Good Thing--I'm volunteering to do all the legwork. :slight_smile:

Please! That would be a very welcome addition!

I asked a similar question here about a month ago and Chris answered:
"make install is lightly tested, but should work. Please report any
problems you find."

Yup, the installed headers and libraries seem sufficient so far. Linking is slow, but that's to be expected for such a big C++ project.

For what it's worth, linking is significantly faster for a release build than a debug build. Also, if you're on linux, updating to a new binutils can help things significantly.

Personally, I've found LLVM to be a pleasure to work with. Its design is very powerful, which simplifies many things a lot.

Oh, you don't have to convince me of that! LLVM's design is so elegant (compared to some compiler backends I've seen) that it really ought to have theme music or something. :slight_smile:

:slight_smile:

-Chris

5

Gnome (and many other Unix projects with massively ugly dependencies) can be linked trivially using a "foo-config" script. If LLVM had something similar, it might save new LLVM developers several hours of digging through manuals and Makefiles.

This would be very very cool to have.

OK! Would something like the following interface be a reasonable first approach?

   llvm-config (--cxxflags | --ldflags | --libs) (all | jit)
     --cxxflags: Flags to use when compiling C++ source code.
     --ldflags: Linker flags to use when linking against LLVM.
     --libs: Libraries needed to link the current configuration.
     all: Link all LLVM libraries for the current platform.
     jit: Link the libraries needed by the standard JIT/interpreter configuration.

This fits nicely into the autoconf/autoconf view of the world, and would only take an evening to get working. If llvm-gcc 4.0 requires a specific subset of libraries, we could add a third hard-coded configuration.

Later on, we could consider adding fine-grained control over which libraries get linked, perhaps based on output from GenLibDeps.pl. But that would require making GenLibDeps.pl more portable (it currently uses some non-portable nm flags). In any case, I'm mostly focused on making things easy for first-time LLVM users.

For what it's worth, linking is significantly faster for a release build than a debug build. Also, if you're on linux, updating to a new binutils can help things significantly.

Thanks for the tips! I've been writing a (very primitive) llvm-grep program as a warmup exercise, and I was able to JIT good code within a day of getting started.

Many thanks for releasing such a cool project!

Cheers,
Eric

6

Gnome (and many other Unix projects with massively ugly dependencies) can be linked trivially using a "foo-config" script. If LLVM had something similar, it might save new LLVM developers several hours of digging through manuals and Makefiles.

This would be very very cool to have.

OK! Would something like the following interface be a reasonable first approach?

llvm-config (--cxxflags | --ldflags | --libs) (all | jit)
   --cxxflags: Flags to use when compiling C++ source code.
   --ldflags: Linker flags to use when linking against LLVM.
   --libs: Libraries needed to link the current configuration.
   all: Link all LLVM libraries for the current platform.
   jit: Link the libraries needed by the standard JIT/interpreter configuration.

Sounds great.

This fits nicely into the autoconf/autoconf view of the world, and would only take an evening to get working. If llvm-gcc 4.0 requires a specific subset of libraries, we could add a third hard-coded configuration.

Ok.

Later on, we could consider adding fine-grained control over which libraries get linked, perhaps based on output from GenLibDeps.pl. But that would require making GenLibDeps.pl more portable (it currently uses some non-portable nm flags). In any case, I'm mostly focused on making things easy for first-time LLVM users.

I would be quite happy to have a hard-coded list of library dependencies, even if it means that we need to manually maintain them. This will force us to realize and think about what dependencies there are between libraries.

For what it's worth, linking is significantly faster for a release build than a debug build. Also, if you're on linux, updating to a new binutils can help things significantly.

Thanks for the tips! I've been writing a (very primitive) llvm-grep program as a warmup exercise, and I was able to JIT good code within a day of getting started.

Cool!

-Chris

7

OK! Would something like the following interface be a reasonable first approach?

llvm-config (--cxxflags | --ldflags | --libs) (all | jit)
   --cxxflags: Flags to use when compiling C++ source code.
   --ldflags: Linker flags to use when linking against LLVM.
   --libs: Libraries needed to link the current configuration.
   all: Link all LLVM libraries for the current platform.
   jit: Link the libraries needed by the standard JIT/interpreter configuration.

Sounds great.

OK! I've implemented a simple version of llvm-config, just to give us something concrete to talk about. :slight_smile: I'll send you the patch in a separate, off-list message. If anybody else would like a copy, just ask.

llvm-config supports the following options:

   --version LLVM version.
   --prefix Installation prefix.
   --bindir Directory containing LLVM executables.
   --includedir Directory containing LLVM headers.
   --libdir Directory containing LLVM libraries.
   --cxxflags C++ compiler flags for file which include LLVM headers.
   --ldflags Linker flags.
   --libs <COMPONENT>... Libraries needed to link against LLVM components.

Right now, the only useful component is the JIT:

   jit LLVM JIT compiler.

I get this set of dependencies using:

   LLVMLIBS := JIT
   SUB_JITOBJS=$(LLVMUsedLibs)

It's very easy to dump a Makefile variable to the llvm-config script, so the only obstacle to adding more configurations is modularizing the code which processes LLVMLIBS.

I would be quite happy to have a hard-coded list of library dependencies, even if it means that we need to manually maintain them. This will force us to realize and think about what dependencies there are between libraries.

This is a good idea. I'm not quite sure how to process a dependency list using /bin/sh, but maybe inspiration will strike me. :slight_smile:

If there's any changes you'd like me to make to this patch, please let me know! This is just a minimalist version to spur further discussion.

Cheers,
Eric

8

Hi Eric,

I do most of the maintenance on the configuration and Makefile stuff.
So, I would like a copy of this to try out.

Thanks,

Reid.