Transitioning build to cmake

I recently took a stab at changing the MacPorts llvm-3.4 port from the configure-based build system to the cmake-based build system.

There are a couple of issues that I still haven't been able to work out yet and would like to know if these are just configuration issues on my side or bugs I should file at bugs.llvm.org:

1) libclang_rt

It looks like the cmake build is missing some runtime libraries. How do I coerce it into building the right ones?

-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.10.4.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.cc_kext.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.eprintf.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ios.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.osx.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_ios.a
+./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.i386.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_osx.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ubsan_osx.a
+./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.x86_64.a

2) Universal

This looks like a bug in cmake. I'm unable to build universal when setting CMAKE_OSX_ARCHITECTURES. cmake errors out because it only set the -arch command line arguments at link time:

  ...
  Building C object
  CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o

  /usr/bin/clang -Os -o
  CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o -c
  /opt/local/var/macports/build/_Users_jeremy_src_macports_trunk_dports_lang_llvm-3.4/llvm-3.4/work/build/CMakeFiles/CMakeTmp/testCCompiler.c

  Linking C executable cmTryCompileExec3905760613

  /opt/local/bin/cmake -E cmake_link_script
  CMakeFiles/cmTryCompileExec3905760613.dir/link.txt --verbose=1

  /usr/bin/clang -Os -Wl,-search_paths_first -Wl,-headerpad_max_install_names
  -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -arch i386
  CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o -o
  cmTryCompileExec3905760613

  ld: warning: ignoring file
  CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o, file was built
  for unsupported file format ( 0xCF 0xFA 0xED 0xFE 0x07 0x00 0x00 0x01 0x03
  0x00 0x00 0x00 0x01 0x00 0x00 0x00 ) which is not the architecture being
  linked (i386): CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o

  Undefined symbols for architecture i386:
  ...

3) Shared library

The build fails if I try to build llvm using BUILD_SHARED_LIBS=ON ... the issue is that when the build tries to use the tools, dyld can't find libllvm-3.4svn.dylib because it's not yet installed.

4) Building clang using installed llvm

It looks like there is some support for building clang against an installed llvm by setting CLANG_PATH_TO_LLVM_BUILD. This fails miserably in part because the installed llvm cmake files reference build time paths, but even after fixing that, there are tons of build failures. I'm guessing this is still a work in progress, but if I should file bugs, please let me know.

I recently took a stab at changing the MacPorts llvm-3.4 port from the
configure-based build system to the cmake-based build system.

Thanks for working on this!

4) Building clang using installed llvm

It looks like there is some support for building clang against an
installed llvm by setting CLANG_PATH_TO_LLVM_BUILD. This fails miserably
in part because the installed llvm cmake files reference build time paths,
but even after fixing that, there are tons of build failures. I'm guessing
this is still a work in progress, but if I should file bugs, please let me
know.

This is probably not a very good idea because clang evolves in lock-step
with LLVM. Unless the installed LLVM is the same revision as the clang you
are building, things are likely to not work due to internal API changes.
The option you cite is more likely intended for when you build clang in a
directory separate from LLVM (rather than when it is in llvm/tools/clang/,
where things just work) but both are still checked out at the same revision.

-- Sean Silva

No prob. Hopefully we can iron out these wrinkles.

They match revision. We just need to split the build into two subports to break dependency cycles, and I’m hoping to avoid rebuilding the core libraries a second time.

Ah, OK. Definitely a bug in our build system then.

-- Sean Silva

I recently took a stab at changing the MacPorts llvm-3.4 port from the configure-based build system to the cmake-based build system.

There are a couple of issues that I still haven't been able to work out yet and would like to know if these are just configuration issues on my side or bugs I should file at bugs.llvm.org:

tl;dr: LLVM CMake support is primarily designed for Windows/Visual Studio (especially in LLVM proper) and Linux (especially in compiler-rt), and needs lots of work to work well on Mac OS X or anywhere else. In particular, it is missing many features that are present in the autotools build. (Though, as the CMake proponents 'round here are quick to point out, the autotools system is itself missing some features that are present in CMake.)

1) libclang_rt

It looks like the cmake build is missing some runtime libraries. How do I coerce it into building the right ones?

-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.10.4.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.cc_kext.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.eprintf.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ios.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.osx.a
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_ios.a
+./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.i386.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_osx.a
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ubsan_osx.a
+./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.x86_64.a

The CMake support in compiler-rt evolved in a completely different direction from the Makefiles; it was primarily designed originally, IIRC, to support building asan (and later, the other sanitizers), and mostly on Linux at that. Other platforms and configurations were an afterthought. It needs serious work--in particular, the runtime libraries are built with the host compiler (not usually a problem, unless you're building a cross compiler), and (as you've probably noticed by now) it doesn't make fat archives. Patches welcome if you can speak CMake ;).

2) Universal

This looks like a bug in cmake. I'm unable to build universal when setting CMAKE_OSX_ARCHITECTURES. cmake errors out because it only set the -arch command line arguments at link time:

...
Building C object
CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o

/usr/bin/clang -Os -o
CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o -c
/opt/local/var/macports/build/_Users_jeremy_src_macports_trunk_dports_lang_llvm-3.4/llvm-3.4/work/build/CMakeFiles/CMakeTmp/testCCompiler.c

Linking C executable cmTryCompileExec3905760613

/opt/local/bin/cmake -E cmake_link_script
CMakeFiles/cmTryCompileExec3905760613.dir/link.txt --verbose=1

/usr/bin/clang -Os -Wl,-search_paths_first -Wl,-headerpad_max_install_names
-L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -arch i386
CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o -o
cmTryCompileExec3905760613

ld: warning: ignoring file
CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o, file was built
for unsupported file format ( 0xCF 0xFA 0xED 0xFE 0x07 0x00 0x00 0x01 0x03
0x00 0x00 0x00 0x01 0x00 0x00 0x00 ) which is not the architecture being
linked (i386): CMakeFiles/cmTryCompileExec3905760613.dir/testCCompiler.c.o

Undefined symbols for architecture i386:
...

Well that's odd. I have CMake from trunk installed, and I was able to use it to build a very simple project with one C source universal. I was also able to build CMake itself universal. There's no bug in CMake--at least, not anymore. There might, however, be a bug in LLVM's build system that's causing this.

3) Shared library

The build fails if I try to build llvm using BUILD_SHARED_LIBS=ON ... the issue is that when the build tries to use the tools, dyld can't find libllvm-3.4svn.dylib because it's not yet installed.

The CMake build obviously needs to be taught to set the install names to relative paths on Mac OS, like the autotools build does.

Chip

I recently took a stab at changing the MacPorts llvm-3.4 port from the configure-based build system to the cmake-based build system.

There are a couple of issues that I still haven't been able to work out yet and would like to know if these are just configuration issues on my side or bugs I should file at bugs.llvm.org:

tl;dr: LLVM CMake support is primarily designed for Windows/Visual Studio (especially in LLVM proper) and Linux (especially in compiler-rt), and needs lots of work to work well on Mac OS X or anywhere else. In particular, it is missing many features that are present in the autotools build. (Though, as the CMake proponents 'round here are quick to point out, the autotools system is itself missing some features that are present in CMake.)

The main reason for this was because it looked like autoconf was the second rate citizen:
  1) only cmake supported building building the ubsan runtime for a while: http://llvm.org/bugs/show_bug.cgi?id=14341
  2) some guides only talk about building with cmake (eg http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html)
  3) etc...

If 2-systems is the long term goal, then I'll stick with what works (but still file bugs if cmake folks want to track them). If autoconf is going away at some point, I'd prefer to make the transition while we still have the fallback rather than have a period without it being available.

...

The CMake support in compiler-rt evolved in a completely different direction from the Makefiles; it was primarily designed originally, IIRC, to support building asan (and later, the other sanitizers), and mostly on Linux at that. Other platforms and configurations were an afterthought. It needs serious work--in particular, the runtime libraries are built with the host compiler (not usually a problem, unless you're building a cross compiler), and (as you've probably noticed by now) it doesn't make fat archives. Patches welcome if you can speak CMake ;).

Ok. Unfortunately, I speak m4 way better than cmake. I've used it a few times, but I don't have much experience with it.

...

Well that's odd. I have CMake from trunk installed, and I was able to use it to build a very simple project with one C source universal. I was also able to build CMake itself universal. There's no bug in CMake--at least, not anymore. There might, however, be a bug in LLVM's build system that's causing this.

Ok, I'll try to figure this one out when I get some cycles. I can probably trace where that goes wrong. FWIW, I have cmake 2.8.10.2.

3) Shared library

The build fails if I try to build llvm using BUILD_SHARED_LIBS=ON ... the issue is that when the build tries to use the tools, dyld can't find libllvm-3.4svn.dylib because it's not yet installed.

The CMake build obviously needs to be taught to set the install names to relative paths on Mac OS, like the autotools build does.

Yep ... probably because as you mentioned above, this was primarily for VisualStudio and Linux as clients.

Thanks for your responses,
Jeremy

I recently took a stab at changing the MacPorts llvm-3.4 port from the configure-based build system to the cmake-based build system.

There are a couple of issues that I still haven't been able to work out yet and would like to know if these are just configuration issues on my side or bugs I should file at bugs.llvm.org:

tl;dr: LLVM CMake support is primarily designed for Windows/Visual Studio (especially in LLVM proper) and Linux (especially in compiler-rt), and needs lots of work to work well on Mac OS X or anywhere else. In particular, it is missing many features that are present in the autotools build. (Though, as the CMake proponents 'round here are quick to point out, the autotools system is itself missing some features that are present in CMake.)

The main reason for this was because it looked like autoconf was the second rate citizen:
1) only cmake supported building building the ubsan runtime for a while: http://llvm.org/bugs/show_bug.cgi?id=14341
2) some guides only talk about building with cmake (eg http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html)
3) etc...

If 2-systems is the long term goal, then I'll stick with what works (but still file bugs if cmake folks want to track them). If autoconf is going away at some point, I'd prefer to make the transition while we still have the fallback rather than have a period without it being available.

About the only thing I can gather that we've been able to agree on is that we do want to get rid of "autohell" (as some people around here like to call it :slight_smile: at some point. In fact, AFAICT we're just keeping it around because some internal Apple build processes (cf. utils/buildit) still need it. Most people around here just use CMake now--especially because then, they can use Ninja to build LLVM--but I seem to recall (and a lot of people seem to have forgotten) Daniel Dunbar talking about replacing both autoconf and CMake with some custom build system that he was going to write in Python; the LLVMBuild.txt stuff was supposed to be the first phase of that. (Then a bunch of people, mostly from *BSD, objected because they didn't want Python as a build-time dependency, even though it already is now. They seem to be doing fine with LLVM despite that.) I also remember Daniel saying that he didn't really like CMake, not least because (at least, at the time he said it) its Xcode support sucked horribly; but at this point, I'm not sure he's ever going to get around to finishing this new build system of his. When push finally comes to shove, we'll probably end up with a CMake-based build system (unless Daniel or someone else pipes up in the meantime about it).

...

The CMake support in compiler-rt evolved in a completely different direction from the Makefiles; it was primarily designed originally, IIRC, to support building asan (and later, the other sanitizers), and mostly on Linux at that. Other platforms and configurations were an afterthought. It needs serious work--in particular, the runtime libraries are built with the host compiler (not usually a problem, unless you're building a cross compiler), and (as you've probably noticed by now) it doesn't make fat archives. Patches welcome if you can speak CMake ;).

Ok. Unfortunately, I speak m4 way better than cmake. I've used it a few times, but I don't have much experience with it.

Unfortunately, I don't have many CPU cycles of my own to devote to fixing this (or else, I might have fixed this already). @LLVMdev: any volunteers?

...

Well that's odd. I have CMake from trunk installed, and I was able to use it to build a very simple project with one C source universal. I was also able to build CMake itself universal. There's no bug in CMake--at least, not anymore. There might, however, be a bug in LLVM's build system that's causing this.

Ok, I'll try to figure this one out when I get some cycles. I can probably trace where that goes wrong. FWIW, I have cmake 2.8.10.2.

I tried v2.8.10.2 on this really small test project that I've attached (the same one I used to verify that CMake HEAD works), and it builds fat just fine. So yeah, there's almost definitely a problem with LLVM CMake and universal builds. Or, maybe there's a local problem with your CMake. (Not likely, but still...) Try building the attached (after un-tar'ing) like so:

$ cd test.cm
$ cmake -DCMAKE_OSX_ARCHITECTURES="i386;x86_64" -G"Unix Makefiles"
$ make

and then you can run lipo -info on the "test" file that should've been built. It should report that there are two architectures in the file, i386 and x86_64.

Thanks for your responses,

No problem.

Chip

test.tar.bz2 (542 Bytes)

We do the same thing in FreeBSD and keeping the ability to build with
the installed llvm is critical given our current ports/package
infrastructure. The build with configure based builds is a bit of a
hack, but seems to break quite infrequently.

-- Brooks

> I recently took a stab at changing the MacPorts llvm-3.4 port from the
configure-based build system to the cmake-based build system.
>
> There are a couple of issues that I still haven't been able to work out
yet and would like to know if these are just configuration issues on my
side or bugs I should file at bugs.llvm.org:
>
>
tl;dr: LLVM CMake support is primarily designed for Windows/Visual Studio
(especially in LLVM proper) and Linux (especially in compiler-rt), and
needs lots of work to work well on Mac OS X or anywhere else. In
particular, it is missing many features that are present in the autotools
build. (Though, as the CMake proponents 'round here are quick to point out,
the autotools system is itself missing some features that are present in
CMake.)

> 1) libclang_rt
>
> It looks like the cmake build is missing some runtime libraries. How do
I coerce it into building the right ones?
>
> -./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.10.4.a
>
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
>
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.cc_kext.a
>
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.eprintf.a
> -./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ios.a
> -./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.osx.a
>
-./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_ios.a
> +./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.i386.a
>
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.profile_osx.a
>
./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.ubsan_osx.a
>
+./opt/local/libexec/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.x86_64.a
The CMake support in compiler-rt evolved in a completely different
direction from the Makefiles; it was primarily designed originally, IIRC,
to support building asan (and later, the other sanitizers), and mostly on
Linux at that. Other platforms and configurations were an afterthought. It
needs serious work--in particular, the runtime libraries are built with the
host compiler (not usually a problem, unless you're building a cross
compiler), and (as you've probably noticed by now) it doesn't make fat
archives. Patches welcome if you can speak CMake ;).

FWIW, we use CMake to build fat ASan runtime on Mac OS X, and it works
pretty well for us. I think that support for at least some of another
compiler-rt runtime libraries on Mac may be added with a reasonable effort
- there just wasn't any demand for that recently. I'm ready to review
patches or add some build rules myself, but as I work primarily on Linux
and don't know Clang use cases in the Mac world, I'll have problems testing
these changes.