Living on Clang

Hello fellow LLVMers and Clangstas,

We want to make Clang great, and we need your help!

Helping is easy: just build Clang on your platform and start using it as your main compiler for LLVM and Clang development. Much of the Clang team has been living on Clang for at least several weeks already, and we've found it to be quite stable for development. If you run into problems---poor diagnostics, poor debugging experience, compiler crashes, rejecting valid code, whatever---please file Bugzillas so we can get them fixed.

Steps to living on Clang:

  - Make sure you're getting a good revision of Clang/LLVM from Subversion by checking the Buildbots (http://google1.osuosl.org:8011/console). The clang-*-selfhost builds have passed self-host + testing and are good candidates.

  - Build Clang as part of LLVM (http://clang.llvm.org/get_started.html). I suggest a Release build, since Debug builds tend to be very slow.

  - Install Clang + LLVM somewhere that isn't where you do your normal LLVM/Clang development.

  - Configure with the environment variables CC=/path/to/clang CXX=/path/to/clang++, which works with either autoconf configure or CMake cmake/ccmake.

  - Build LLVM and Clang as normal.

  - Report your experiences to help us improve Clang!

  - Doug

I'd love to, but this bug is a blocker for me:
http://llvm.org/bugs/show_bug.cgi?id=6795

Best regards,
--Edwin

Thanks; we didn't know it was blocking anyone. I've marked it as such and we'll try to get it fixed.

  - Doug

I've been hoping to use Clang both for Mac OS X/iPhone OS development, as well as for embedded ARM development. My ARM tools currently are arm-elf-gcc and friends.

Is it possible to build Clang & LLVM to target this platform? Note that I don't actually need ELF, it's an intermediate step toward a raw binary that gets downloaded to the device.

I've been trying, with very limited success, to target ARM. I'm still not clear on where the divisions in the process really lie. What I'm hoping for (someday) is a "GNU-free" toolsuite based on Clang and LLVM. But for now, I can live with arm-elf-as and -ld, if that's the right way to go. I still don't know how LTO works into that.

TIA,

I switched to Clang, and I got 49 new unexpected failures on Linux. Is
this normal? (New, as in, there were no unexpected failures when
compiled the same source with GCC.)

Hi,

Hello fellow LLVMers and Clangstas,

We want to make Clang great, and we need your help!

Helping is easy: just build Clang on your platform and start using it as your main compiler for LLVM and Clang development. Much of the Clang team has been living on Clang for at least several weeks already, and we've found it to be quite stable for development. If you run into problems---poor diagnostics, poor debugging experience, compiler crashes, rejecting valid code, whatever---please file Bugzillas so we can get them fixed.

Been there, done that: René Rebe » Blog Archive » LLVM clang support in T2 SDE (Linux)

And reported C++ issues spotted while at it. I plan to further extend the testing and potentially default to Clang (without GCC) in a future T2 Linux revision.

This is not normal. It may be a symptom of

  http://llvm.org/bugs/show_bug.cgi?id=6795

It looks like we still need some help tracking down problems on Linux; we're not seeing any of these failures on Mac OS X.

  - Doug

I can't switch to clang on my project until it can handle boost headers.

I can't switch to clang on my project until it can handle boost headers.

The intent of using Clang for LLVM & Clang development is that we can improve compiler quality (diagnostics, debugging, etc.) on a code base where Clang works reasonably well already.

For other code bases, we're likely to have more basic problems---rejecting valid code, complaining about ill-formed code that GCC accepts, horrid miscompilations, etc.---so we get a different kind of feedback. For Boost specifically, we have a tracking bug (http://llvm.org/bugs/show_bug.cgi?id=6023) and are interested in reduced problems that show up in Boost libraries [*].

  - Doug

[*] Actually, we're *particularly* interested in Boost since there will be a Clang talk at BoostCon this year. Check it out at BoostCon.com is for sale | HugeDomains .

I never bother installing clang - I just add the build location to my path, which saves some time and disk space. Unfortunately, this makes it impossible to build clang with clang, because it's replacing the libraries and executable while building.

Would it be possible for the LLVM / Clang makefiles to build these files with a temporary name and then move them over the originals when the build process is finished? This would make it much easier to live on clang.

I also had problems last time I tried building LLVM with clang - apparently the makefiles were not respecting the CXX environment variable, so you need a complete reconfigure, which is incredibly irritating. With other projects I test with GCC and clang just by doing:

$ gmake CC=clang
$ gmake clean
$ gmake CC=gcc

David

-- Sent from my STANTEC-ZEBRA

Helping is easy: just build Clang on your platform and start using it as your main compiler for LLVM and Clang development

I never bother installing clang - I just add the build location to my path, which saves some time and disk space. Unfortunately, this makes it impossible to build clang with clang, because it's replacing the libraries and executable while building.

I suggest keeping a separate Clang-built-Clang that you live on, and update periodically to known-good revisions. Otherwise, you're likely to end up with a broken host compiler when something goes wrong on the trunk, which is a complete waste of time. Personally, I update my host Clang each morning to a version that has passed self-host + test.

Would it be possible for the LLVM / Clang makefiles to build these files with a temporary name and then move them over the originals when the build process is finished? This would make it much easier to live on clang.

A top level "bootstrap" target would address this issue better. I'd be opposed to having the normal build create temporary compilers that get moved over the originals, because that's more build-system ugliness to confuse IDEs and users alike.

I also had problems last time I tried building LLVM with clang - apparently the makefiles were not respecting the CXX environment variable, so you need a complete reconfigure, which is incredibly irritating. With other projects I test with GCC and clang just by doing:

$ gmake CC=clang
$ gmake clean
$ gmake CC=gcc

gmake CXX=clang++ CC=clang

should work fine with the normal makefiles.

  - Doug

We want to make Clang great, and we need your help!

Doug,

I don't see any type of selfbuild target in the LLVM top level makefile.
This would be a useful way to automate the self build process.

I've used similar mechanisms before on other compilers, where you can
trivially invoke a three phase build (first you do a full build with your
system compiler, then you use your built compiler to do another full build,
and then take that most recent build compiler to build yet again...). The
build options of the last two phases are identical, and the object files
from those last two passes are compared to ensure that they are identical.

Ideally you can control the build options of the initial build and last two
builds independently (e.g. build a compiler without optimizations and with
assertions, and then do the last two builds with assertions disabled, and
optimizations on) - I'm not sure how easy this is with the autoconfig based
system LLVM/clang use, if at all.

Mark

Yes, we would love to have this, but nobody has taken the time to write the Makefile/CMake magic to make it happen.

  - Doug

How about building this in the CMake or auto* level we build a separate build script for it? I attache a simple version here. It does seem to work here. It’s a shell script, which uses cmake for the building but it seems the second stage file works. I am not really sure why a third stage is needed, but it can be added quite easily. A windows version should be trivial to write (next week).

A few rants about clang:
This is the first time I was able to bootstrap the whole compiler. Which is great, don’t get me wrong. But it was painfully slow, and the CPU of my machine became really hot unlike with gcc. The whole building process took more then 4 hours on this old cual core machine, while stage one took an hour more or less (I need to do a test again, without ccache).

[elcuco@pinky ~/src/llvm] ls bootstrap-stage-2/bin/ -ltr
total 262708
-rwxrwxr-x. 1 elcuco elcuco 5641794 2010-04-19 03:12 tblgen
-rwxrwxr-x. 1 elcuco elcuco 985416 2010-04-19 04:41 FileCheck
-rwxrwxr-x. 1 elcuco elcuco 6374 2010-04-19 04:41 count
-rwxrwxr-x. 1 elcuco elcuco 122992 2010-04-19 04:41 not
-rwxrwxr-x. 1 elcuco elcuco 24101 2010-04-19 05:34 llvm-config
-rwxrwxr-x. 1 elcuco elcuco 18325933 2010-04-19 05:36 opt
-rwxrwxr-x. 1 elcuco elcuco 7705301 2010-04-19 05:36 llvm-as
-rwxrwxr-x. 1 elcuco elcuco 6671138 2010-04-19 05:36 llvm-dis
-rwxrwxr-x. 1 elcuco elcuco 31626605 2010-04-19 05:37 llvm-mc
-rwxrwxr-x. 1 elcuco elcuco 32630459 2010-04-19 05:38 llc
-rwxrwxr-x. 1 elcuco elcuco 6814656 2010-04-19 05:38 llvm-ranlib
-rwxrwxr-x. 1 elcuco elcuco 6875243 2010-04-19 05:38 llvm-ar
-rwxrwxr-x. 1 elcuco elcuco 6827605 2010-04-19 05:38 llvm-nm
-rwxrwxr-x. 1 elcuco elcuco 14608549 2010-04-19 05:39 llvm-ld
-rwxrwxr-x. 1 elcuco elcuco 7506306 2010-04-19 05:39 llvm-prof
-rwxrwxr-x. 1 elcuco elcuco 8310633 2010-04-19 05:39 llvm-link
-rwxrwxr-x. 1 elcuco elcuco 25527057 2010-04-19 05:40 lli
-rwxrwxr-x. 1 elcuco elcuco 8217315 2010-04-19 05:40 llvm-extract
-rwxrwxr-x. 1 elcuco elcuco 18994379 2010-04-19 05:42 bugpoint
-rwxrwxr-x. 1 elcuco elcuco 1011333 2010-04-19 05:43 llvm-bcanalyzer
-rwxrwxr-x. 1 elcuco elcuco 6468 2010-04-19 05:43 llvm-stub
-rwxrwxr-x. 1 elcuco elcuco 42747 2010-04-19 07:28 c-index-test
-rwxrwxr-x. 1 elcuco elcuco 60447800 2010-04-19 07:29 clang
lrwxrwxrwx. 1 elcuco elcuco 51 2010-04-19 07:29 clang++ → /home/elcuco/src/llvm/bootstrap-stage-2/bin/./clang

bootstrap-clang.sh (372 Bytes)

If this is a dual core, you may try

make -j2

to boost the build.

How about building this in the CMake or auto* level we build a separate build script for it? I attache a simple version here. It does seem to work here. It’s a shell script, which uses cmake for the building but it seems the second stage file works. I am not really sure why a third stage is needed, but it can be added quite easily. A windows version should be trivial to write (next week).

A few rants about clang:
This is the first time I was able to bootstrap the whole compiler. Which is great, don’t get me wrong. But it was painfully slow, and the CPU of my machine became really hot unlike with gcc. The whole building process took more then 4 hours on this old cual core machine, while stage one took an hour more or less (I need to do a test again, without ccache).

[elcuco@pinky ~/src/llvm] ls bootstrap-stage-2/bin/ -ltr
total 262708
-rwxrwxr-x. 1 elcuco elcuco 5641794 2010-04-19 03:12 tblgen
-rwxrwxr-x. 1 elcuco elcuco 985416 2010-04-19 04:41 FileCheck
-rwxrwxr-x. 1 elcuco elcuco 6374 2010-04-19 04:41 count
-rwxrwxr-x. 1 elcuco elcuco 122992 2010-04-19 04:41 not
-rwxrwxr-x. 1 elcuco elcuco 24101 2010-04-19 05:34 llvm-config
-rwxrwxr-x. 1 elcuco elcuco 18325933 2010-04-19 05:36 opt
-rwxrwxr-x. 1 elcuco elcuco 7705301 2010-04-19 05:36 llvm-as
-rwxrwxr-x. 1 elcuco elcuco 6671138 2010-04-19 05:36 llvm-dis
-rwxrwxr-x. 1 elcuco elcuco 31626605 2010-04-19 05:37 llvm-mc
-rwxrwxr-x. 1 elcuco elcuco 32630459 2010-04-19 05:38 llc
-rwxrwxr-x. 1 elcuco elcuco 6814656 2010-04-19 05:38 llvm-ranlib
-rwxrwxr-x. 1 elcuco elcuco 6875243 2010-04-19 05:38 llvm-ar
-rwxrwxr-x. 1 elcuco elcuco 6827605 2010-04-19 05:38 llvm-nm
-rwxrwxr-x. 1 elcuco elcuco 14608549 2010-04-19 05:39 llvm-ld
-rwxrwxr-x. 1 elcuco elcuco 7506306 2010-04-19 05:39 llvm-prof
-rwxrwxr-x. 1 elcuco elcuco 8310633 2010-04-19 05:39 llvm-link
-rwxrwxr-x. 1 elcuco elcuco 25527057 2010-04-19 05:40 lli
-rwxrwxr-x. 1 elcuco elcuco 8217315 2010-04-19 05:40 llvm-extract
-rwxrwxr-x. 1 elcuco elcuco 18994379 2010-04-19 05:42 bugpoint
-rwxrwxr-x. 1 elcuco elcuco 1011333 2010-04-19 05:43 llvm-bcanalyzer
-rwxrwxr-x. 1 elcuco elcuco 6468 2010-04-19 05:43 llvm-stub
-rwxrwxr-x. 1 elcuco elcuco 42747 2010-04-19 07:28 c-index-test
-rwxrwxr-x. 1 elcuco elcuco 60447800 2010-04-19 07:29 clang
lrwxrwxrwx. 1 elcuco elcuco 51 2010-04-19 07:29 clang++ → /home/elcuco/src/llvm/bootstrap-stage-2/bin/./clang

We want to make Clang great, and we need your help!

Doug,

I don’t see any type of selfbuild target in the LLVM top level makefile.
This would be a useful way to automate the self build process.

I’ve used similar mechanisms before on other compilers, where you can
trivially invoke a three phase build (first you do a full build with your
system compiler, then you use your built compiler to do another full build,
and then take that most recent build compiler to build yet again…). The
build options of the last two phases are identical, and the object files
from those last two passes are compared to ensure that they are identical.

Ideally you can control the build options of the initial build and last two
builds independently (e.g. build a compiler without optimizations and with
assertions, and then do the last two builds with assertions disabled, and
optimizations on) - I’m not sure how easy this is with the autoconfig based
system LLVM/clang use, if at all.

Mark


LLVM Developers mailing list
LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

<bootstrap-clang.sh>_______________________________________________
cfe-dev mailing list
cfe-dev@cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

– Jean-Daniel

Diego Iastrubni <diegoiast@gmail.com>
writes:

How about building this in the CMake or auto* level we build a separate
build script for it?

I was thinking along the same lines, although a cmake script will do
too, the only tricky point being the special syntax required for
executing it.

I attache a simple version here. It does seem to work
here. It's a shell script, which uses cmake for the building but it seems
the second stage file works. I am not really sure why a third stage is
needed, but it can be added quite easily.

The third stage is for comparing the output of clang (as compiled by
gcc) against clang (as compiled by clang). The whole process is:

Stage 1: build clang with gcc

Stage 2: build clang with the clang created by gcc

Stage 3: build clang with the clang created by clang.

Final test: compare the object files geneated on Stage 2 with the object
files generated on Stage 3.

The last step is useful for detecting bugs on clang (and on gcc).

A windows version should be trivial to write (next week).

Comparing the object files on Windows may be tricky, because IIRC they
contain timestamps.

A few rants about clang:
This is the first time I was able to bootstrap the whole compiler. Which is
great, don't get me wrong. But it was painfully slow, and the CPU of my
machine became *really* hot unlike with gcc. The whole building process took
more then 4 hours on this old cual core machine, while stage one took an
hour more or less (I need to do a test again, without ccache).

Building llvm+clang requires less than 10 minutes here with an Intel
Q6600, Linux x86_64.

> A windows version should be trivial to write (next week).

Comparing the object files on Windows may be tricky, because IIRC they
contain timestamps.

This could be worked around by adding a hack (enabled by command-line
options of various tools) to zero out the time/date stamp in the MS COFF
header.

Mark

How about building this in the CMake or auto* level we build a separate build script for it? I attache a simple version here. It does seem to work here. It's a shell script, which uses cmake for the building but it seems the second stage file works. I am not really sure why a third stage is needed, but it can be added quite easily. A windows version should be trivial to write (next week).

We can do this with a script, although it would be more convenient if it were a part of the build system itself. Regarding the script itself...

mkdir -p bootstrap-stage-1
cd bootstrap-stage-1
cmake -DVARIABLE:CMAKE_BUILD_TYPE=release ../
make VERBOSE=1
cd ..

Your CMake command-line syntax is wrong. To set CMAKE_BUILD_TYPE to Release, use

cmake -DCMAKE_BUILD_TYPE:STRING=Release ../

A few rants about clang:
This is the first time I was able to bootstrap the whole compiler. Which is great, don't get me wrong. But it was painfully slow, and the CPU of my machine became *really* hot unlike with gcc. The whole building process took more then 4 hours on this old cual core machine, while stage one took an hour more or less (I need to do a test again, without ccache).

That's what happens when you use an unoptimized Clang to build Clang :slight_smile: Your mis-typed setting of CMAKE_BUILD_TYPE meant that the first stage built a debug Clang. I'm sure things will go better when you turn optimization on.

  - Doug

I took your suggestion, and it's worked out pretty well so far.

I did not quite understand how to add my own system header paths to the hard-coded paths in clang/lib/Frontend/InitHeaderSearch.cpp (my attempt failed to solve the problem), so I ended up having to create a link from /usr/include/stdarg.h to /usr/lib/gcc/i686-apple-darwin9/4.0.1/include/stdarg.h so that <iostream> would compile.

In my latest small project (51 headers, 14 sources so far), Clang was able to compile every source, some of which included templates, except for the ones that instantiated boost::uuids::string_generator::operator() (Boost.Uuid is a new library in version 1.42). However, GCC was able to compile those sources and successfully link the resulting objects with the Clang-built objects.

Knowing where Clang has come from, I am impressed with the progress! I'm looking forward to the day I can completely switch over. :slight_smile:

- John

Hrm. The Boost.UUID tests are passing using Boost from trunk (and yesterday's Clang from trunk). Did you file a bug about the failure you're seeing?

  - Doug