Building llvm so it can be installed by other users

David_Lobron · October 20, 2015, 8:26pm

Hey All,

I'm working on packaging the current version of llvm for use by other people at my company. As part of this, I'd like to make a tarball of my build, so that other users do not have to rebuild from source in order to use llvm for their builds. Ideally, I'd like my makefile to un-tar my build output, and then do a "make install" to install anything that has been updated - we have a standard install root called "common" that lives one directory up from the build directory on our build machines.

The tricky thing I'm running into is that all of the makefiles generated by cmake contain absolute paths, e.g.,

"/home/dlobron/build/clangport/akamai/llvm/llvm-3.7/llvm/include/llvm"

I was wondering if it's possible to build llvm in such a way that all of these files contain relative paths. It would be perfect for me if the above line, and all other generated makefile paths, would read like this:

"../common/include/llvm"

I tried setting -DCMAKE_INSTALL_PREFIX, but it picked up the full path- I wasn't sure how to tell it to keep the path relative.

Thanks for any help you can give!

--David

jroelofs · October 20, 2015, 8:37pm

Hey All,

I'm working on packaging the current version of llvm for use by other
people at my company. As part of this, I'd like to make a tarball of
my build,

Build directories aren't easily redistributable, nor are they really meant to be. Install directories on the other hand can be, provided you build with the right options, and package together all the dependencies.

Is there a particular reason why you want to distribute your build dir instead of your install dir?

Jon

David_Lobron · October 21, 2015, 1:45pm

Hi Jon,

Build directories aren't easily redistributable, nor are they really meant to be. Install directories on the other hand can be, provided you build with the right options, and package together all the dependencies.

Is there a particular reason why you want to distribute your build dir instead of your install dir?

Oops, you are right: I think it's the install directory that I want to be rooted in a relative directory path. My goal here is for other users to be able to install and use clang and the required headers and libraries, without having to build from scratch. The install path for a given user would be, e.g.,

/home/USERNAME/build/common/

So for example, clang would be installed to /home/USERNAME/build/common/bin.

When I did my initial build, I set -DCMAKE_INSTALL_PREFIX to the "common" path above, but the Makefiles and cmake_makefiles that were generated have a full path (/home/dlobron/build/common), which of course is not portable to other users if they were to do a "make install". Is it possible to set DCMAKE_INSTALL_PREFIX (or another variable) such that I could package my install directory and another user could use it to install to their /home/OTHER-USERNAME/build/common root? Can I get it to write a relative path in the generated makefiles? I checked CMake.html in docs, but I did not see mention of this.

Please also let me know if I appear to be thinking about this the wrong way- I've never attempted to package up a compiler collection for others' use before and I might be missing something obvious.

Thanks!

--David

jroelofs · October 21, 2015, 5:38pm

Hi Jon,

Build directories aren't easily redistributable, nor are they
really meant to be. Install directories on the other hand can be,
provided you build with the right options, and package together all
the dependencies.

Is there a particular reason why you want to distribute your build
dir instead of your install dir?

Oops, you are right: I think it's the install directory that I want
to be rooted in a relative directory path. My goal here is for other
users to be able to install and use clang and the required headers
and libraries, without having to build from scratch. The install
path for a given user would be, e.g.,

/home/USERNAME/build/common/

So for example, clang would be installed to
/home/USERNAME/build/common/bin.

When I did my initial build, I set -DCMAKE_INSTALL_PREFIX to the
"common" path above, but the Makefiles and cmake_makefiles that were
generated have a full path (/home/dlobron/build/common), which of

Don't worry about the paths in the makefiles.

What's important is to make sure that the resulting binaries are relocatable. There are two ways to do that:

     1) Build everything statically linked.
         * If you do this, your binaries will be a lot bigger, but the
             dependency management should be simpler.
         * It is easier to get this one working.
         * To do this, set:
             LDFLAGS="-L/usr/lib -static-libstdc++ -static-ligbcc"

     2) Package up all the dependencies so that the dynamic linker
         knows to find them in a relative path, rather than an absolute
         path.
         * There's two ways to make this one work:
             * By setting an environment variable in your users'
                  environment:
                 `export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH`
                  Where $PREFIX is where you've installed this on your
                  user's system, not necessarily the same as
                  your CMAKE_INSTALL_PREFIX.
             * By arranging for that at compiler compile time:
                  CFLAGS="-Wl,-rpath=../lib"

Once you've got it built, and tested, then you should use `make install` to package everything up into whatever folder you set CMAKE_INSTALL_PREFIX to. That folder is what you should redistribute to your users (via tar, or otherwise). You can check that it worked by moving the folder somewhere else, and seeing if the executables will still run.

course is not portable to other users if they were to do a "make
install". Is it possible to set DCMAKE_INSTALL_PREFIX (or another
variable) such that I could package my install directory and another
user could use it to install to their
/home/OTHER-USERNAME/build/common root?

Yeah, see above.

Can I get it to write a relative path in the generated makefiles?

No, but you don't need it to. See above.

I checked CMake.html in docs, but I did not see mention of this.

This would be a useful addition to the docs.

Please also let me know if I appear to be thinking about this the

Yes. Let me reiterate: your users should not be running `make install` on the makefiles generated by cmake... That's something you'll do once before packaging everything up for them.

wrong way- I've never attempted to package up a compiler collection
for others' use before and I might be missing something obvious.

happy hacking!

Jon

Steve_King · October 21, 2015, 5:56pm

If you go with dynamic linking, using rpath is preferred over LD_LIBRARY_PATH.
https://www.google.com/search?q=LD_LIBRARY_PATH+problems

David_Lobron · October 22, 2015, 2:43pm

Thanks, Jonathan- this worked like a charm.

I checked CMake.html in docs, but I did not see mention of this.

This would be a useful addition to the docs.

I would be glad to add instructions to that page. I'll give that a try!

Thank you again,

David

Tyler_Hardin1 · October 23, 2015, 6:00am

You could just tar up the files that will be installed and write a script to put them in their respective subdirs in /usr/local. (You could probably even steal their install script from the “install” target in their makefile.) If you look in the build directory (or it might be some subdir thereof*), there’s usually a 1-1 correspondence between the folder names and the names of the folders where the files go in /usr or /usr/local. (Of course, you should check for conflicts/overwrites.)

*: It’s been a while since I’ve compiled LLVM. I just follow the list out of interest in their progress.

Bruce_Hoult · October 23, 2015, 9:57am

The usual way to handle this is:

The “–prefix” argument or PREFIX env variable is used to specify the location in which the final program/libraries etc will be stored.

The “–destdir” argument or DESTDIR env variable is used to specify the location in which the build products will be placed when you do “make install”. It will probably not be possible to run the program from this location, because it will look for libraries at the “PREFIX” location.

This makes it easy to simply tar up the entire contents of DESTDIR and then untar them in PREFIX on the same or other machines. Untarring does not affect existing files in the tree you are untarring into.

Chris_Bieneman · October 23, 2015, 5:31pm

Not sure what OS you are distributing for, but on OS X you can use the CMake option LLVM_CREATE_XCODE_TOOLCHAIN, which will create a “Toolchains” folder at your install prefix. That folder will contain a “.xctoolchain” folder, which can be tar’d and distributed.

Users can use it by setting the environment variable EXTERNAL_TOOLCHAINS_DIR to the path of a directory containing .xctoolchain folders, and setting the environment variable TOOLCHAINS to the toolchain identifier (which defaults to org.llvm.3.8.0svn)

-Chris