Building llvm from source with clang instead of gcc

Is it possible to build llvm with a previous version of clang installed
as a binary ? If so, how do I get llvm/configure to use the binary
version of clang rather than gcc to build the latest version of llvm ?

Set environment variables CC and CXX to point to the clang/clang++ binaries before running configure.

-Krzysztof

Do I give the full path to the executable, or just the executable name
if it is already in the PATH, for those environment variables ?

It would have been much better if the environment variables were named
LLVM_CC and LLVM_CXX for the obvious reason that CC and CXX are very
common names.

I actually found out that llvm/configure will choose clang if it is
found in the path. This contradicts what is written at
http://llvm.org/docs/GettingStarted.html#requirements, where it says it
will use the first GCC it finds. Perhaps that explanation needs to be
updated.

Finally I found that if llvm/configure uses clang instead of gcc, it
invariably fails on my Linux distros because it is using the gcc
distributed included directories and an earlier version of clang will
often fail with these gcc header files. Once I force llvm/configure to
use gcc, building llvm/clang succeeds again.

Actually, the variables CC and CXX are used exactly as they are always used: to set the C and C++ compiler. Frankly, I don't see why you would prefer to use different ones.

-Krzysztof

I actually found out that llvm/configure will choose clang if it is
found in the path. This contradicts what is written at
http://llvm.org/docs/GettingStarted.html#requirements, where it says it
will use the first GCC it finds. Perhaps that explanation needs to be
updated.

Updated docs in r171630.

Finally I found that if llvm/configure uses clang instead of gcc, it
invariably fails on my Linux distros because it is using the gcc
distributed included directories and an earlier version of clang will
often fail with these gcc header files. Once I force llvm/configure to
use gcc, building llvm/clang succeeds again.

Right. Maybe we need to skip old clang versions in this case.

Dmitri

I did not realize that this was a standard use of the CC and CXX environment variables under Linux.

I think this should be done until clang can use its own header files and libraries, when it is installed as a binary, rather than the ones that gcc installs. In fact I am a bit amazed that clang does not already have its own header files and libraries for use when one tries llvm/configure and it sets up llvm's build system to use clang as its compiler. Is this the fault of clang or is it the fault of the llvm/configure system ?

I actually found out that llvm/configure will choose clang if it is
found in the path. This contradicts what is written at
http://llvm.org/docs/**GettingStarted.html#**requirements<http://llvm.org/docs/GettingStarted.html#requirements>,
where it says it
will use the first GCC it finds. Perhaps that explanation needs to be
updated.

Updated docs in r171630.

Finally I found that if llvm/configure uses clang instead of gcc, it

invariably fails on my Linux distros because it is using the gcc
distributed included directories and an earlier version of clang will
often fail with these gcc header files. Once I force llvm/configure to
use gcc, building llvm/clang succeeds again.

Right. Maybe we need to skip old clang versions in this case.

I think this should be done until clang can use its own header files and
libraries, when it is installed as a binary, rather than the ones that gcc
installs. In fact I am a bit amazed that clang does not already have its
own header files and libraries for use when one tries llvm/configure and it
sets up llvm's build system to use clang as its compiler. Is this the fault
of clang or is it the fault of the llvm/configure system ?

There is no complete solution for LLVM-only headers/libraries yet on Linux.
For C++, you can use libc++, but you still need some GCC libraries (or
alternatives). And you will at least need glibc or another compatible C
runtime.

Should not what it is needed by clang to work correctly under Linux be posted somewhere on the clang Getting Started web page ? Even what you posted above requires guesswork on my part.

Am I supposed to install libc++ and glibc in a Linux distro for clang to work correctly ?

Which versions do I need ?

How do I tell clang what to use or is this automatically figured out by clang ?

Will llvm/configure use these libraries if I let it choose a clang binary in order to build the latest llvm/clang from source ?

            I actually found out that llvm/configure will choose clang
            if it is
            found in the path. This contradicts what is written at
            http://llvm.org/docs/**GettingStarted.html#**requirements<http://llvm.org/docs/__GettingStarted.html#__requirements>
            <http://llvm.org/docs/**GettingStarted.html#**requirements<http://llvm.org/docs/GettingStarted.html#requirements>
>,

            where it says it
            will use the first GCC it finds. Perhaps that explanation
            needs to be
            updated.

        Updated docs in r171630.

            Finally I found that if llvm/configure uses clang instead of
            gcc, it
            invariably fails on my Linux distros because it is using the
gcc
            distributed included directories and an earlier version of
            clang will
            often fail with these gcc header files. Once I force
            llvm/configure to
            use gcc, building llvm/clang succeeds again.

        Right. Maybe we need to skip old clang versions in this case.

    I think this should be done until clang can use its own header files
    and libraries, when it is installed as a binary, rather than the
    ones that gcc installs. In fact I am a bit amazed that clang does
    not already have its own header files and libraries for use when one
    tries llvm/configure and it sets up llvm's build system to use clang
    as its compiler. Is this the fault of clang or is it the fault of
    the llvm/configure system ?

There is no complete solution for LLVM-only headers/libraries yet on
Linux. For C++, you can use libc++, but you still need some GCC
libraries (or alternatives). And you will at least need glibc or
another compatible C runtime.

Should not what it is needed by clang to work correctly under Linux be
posted somewhere on the clang Getting Started web page ? Even what you
posted above requires guesswork on my part.

Clang will attempt to use your installed system libraries, usually
libstdc++ and glibc on a Linux distribution. We could probably maintain a
libstdc++ compatibility table that says clang x.y is compatible with
libstdc++ up to z.w. In the vast majority of cases, this just works if you
install a somewhat-newer version of clang. The problems arise when you use
an older clang with a newer libstdc++ (e.g. newer gcc and/or linux
distribution).

Am I supposed to install libc++ and glibc in a Linux distro for clang to
work correctly ?

glibc will already be there. libc++ is purely optional. Clang will
happily use libstdc++ (which comes with your linux distribution), but newer
libstdc++ versions may use new GCC extensions that clang does not yet
support. This is an unfortunate condition that arises from depending on
GNU libraries that assume GCC. At this time, libc++ is not entirely
self-contained on Linux and still requires some GCC libraries for some
external functions.

Which versions do I need ?

I'm guessing you are running into issues with the C++ standard library
headers. In that case, it depends on which clang version you are using.
For compatibility reasons, it is best to always use the latest released
clang (if not trunk). If you try again with 3.2, I bet your problems go
away.

How do I tell clang what to use or is this automatically figured out by
clang ?

There are ways to override the location for the headers/libraries of
libstdc++, but I would not recommend this. Upgrading to a newer clang
should be easier in the long run.

Will llvm/configure use these libraries if I let it choose a clang binary
in order to build the latest llvm/clang from source ?

Which libraries? An older libstdc++? You can use CXXFLAGS and LDFLAGS to
adjust the library search paths, just as you use CXX to set the compiler to
use.

Generally, it is recommended to bootstrap clang:

1. Build llvm/clang with system gcc
2. Build llvm/clang with new clang
3. Install new clang somewhere (which will automatically use system
libstdc++/glibc).

This should solve your problem.

        On Sat, Jan 5, 2013 at 5:59 PM, Edward Diener
        <eldlistmailingz@tropicsoft.__com
        <mailto:eldlistmailingz@tropicsoft.com>
        <mailto:eldlistmailingz@__tropicsoft.com
        <mailto:eldlistmailingz@tropicsoft.com>>>

        wrote:

                     I actually found out that llvm/configure will
        choose clang
                     if it is
                     found in the path. This contradicts what is written at
        http://llvm.org/docs/____GettingStarted.html#____requirements
        <http://llvm.org/docs/__GettingStarted.html#__requirements>

        <http://llvm.org/docs/__GettingStarted.html#__requirements
        <http://llvm.org/docs/GettingStarted.html#requirements>>,

                     where it says it
                     will use the first GCC it finds. Perhaps that
        explanation
                     needs to be
                     updated.

                 Updated docs in r171630.

                     Finally I found that if llvm/configure uses clang
        instead of
                     gcc, it
                     invariably fails on my Linux distros because it is
        using the gcc
                     distributed included directories and an earlier
        version of
                     clang will
                     often fail with these gcc header files. Once I force
                     llvm/configure to
                     use gcc, building llvm/clang succeeds again.

                 Right. Maybe we need to skip old clang versions in
        this case.

             I think this should be done until clang can use its own
        header files
             and libraries, when it is installed as a binary, rather
        than the
             ones that gcc installs. In fact I am a bit amazed that
        clang does
             not already have its own header files and libraries for use
        when one
             tries llvm/configure and it sets up llvm's build system to
        use clang
             as its compiler. Is this the fault of clang or is it the
        fault of
             the llvm/configure system ?

        There is no complete solution for LLVM-only headers/libraries yet on
        Linux. For C++, you can use libc++, but you still need some GCC
        libraries (or alternatives). And you will at least need glibc or
        another compatible C runtime.

    Should not what it is needed by clang to work correctly under Linux
    be posted somewhere on the clang Getting Started web page ? Even
    what you posted above requires guesswork on my part.

Clang will attempt to use your installed system libraries, usually
libstdc++ and glibc on a Linux distribution. We could probably maintain
a libstdc++ compatibility table that says clang x.y is compatible with
libstdc++ up to z.w. In the vast majority of cases, this just works if
you install a somewhat-newer version of clang. The problems arise when
you use an older clang with a newer libstdc++ (e.g. newer gcc and/or
linux distribution).

This is what evidently happens. A clang binary distribution gets created for a particular Linux distro but there is no compatibility in the package which states that it only works with some version of libstdc++ or glibc or earlier. Then a later version of gcc gets installed and with it a later version of libstdc++ and glibc as a dependency. Now the binary version of clang no longer works on that system because of the updates to the C/C++ standard library.

I do not believe this should ever happen. I am not focusing blame but simply saying that from an end-user's POV this is not acceptable if one wants to use clang without having to build it oneself.

    Am I supposed to install libc++ and glibc in a Linux distro for
    clang to work correctly ?

glibc will already be there. libc++ is purely optional. Clang will
happily use libstdc++ (which comes with your linux distribution), but
newer libstdc++ versions may use new GCC extensions that clang does not
yet support. This is an unfortunate condition that arises from
depending on GNU libraries that assume GCC. At this time, libc++ is not
entirely self-contained on Linux and still requires some GCC libraries
for some external functions.

    Which versions do I need ?

I'm guessing you are running into issues with the C++ standard library
headers. In that case, it depends on which clang version you are using.
  For compatibility reasons, it is best to always use the latest
released clang (if not trunk). If you try again with 3.2, I bet your
problems go away.

    How do I tell clang what to use or is this automatically figured out
    by clang ?

There are ways to override the location for the headers/libraries of
libstdc++, but I would not recommend this. Upgrading to a newer clang
should be easier in the long run.

    Will llvm/configure use these libraries if I let it choose a clang
    binary in order to build the latest llvm/clang from source ?

Which libraries? An older libstdc++? You can use CXXFLAGS and LDFLAGS
to adjust the library search paths, just as you use CXX to set the
compiler to use.

Generally, it is recommended to bootstrap clang:

1. Build llvm/clang with system gcc
2. Build llvm/clang with new clang
3. Install new clang somewhere (which will automatically use system
libstdc++/glibc).

This should solve your problem.

OK, this sounds good. Can this process not be explained on the clang Getting Started page for those who are building clang from source. I am not sure how step 3) is supposed to go. Do I just copy the built bin directory somewhere else and then put it first in my PATH, before running llvm/configure ? Or is "Install new clang" more complicated than that ?

Originally I thought I could have both an earlier binary version of clang as well as the latest built version of clang in a Linux distro so that I could test code on both for compatibility of my library for end-users. But with the fairly common breakage of the binary version package of clang when libstdc++/glibc gets updated, this has become much less of a reality.

Do you mean that it is recommended to bootstrap clang (i.e. steps 1, 2, 3), as opposed to just building it with whatever compiler that is available (steps 1, 3)? If so, why?

-Krzysztof

Edward Diener <eldlistmailingz@tropicsoft.com> writes:

Clang will attempt to use your installed system libraries, usually
libstdc++ and glibc on a Linux distribution. We could probably maintain
a libstdc++ compatibility table that says clang x.y is compatible with
libstdc++ up to z.w. In the vast majority of cases, this just works if
you install a somewhat-newer version of clang. The problems arise when
you use an older clang with a newer libstdc++ (e.g. newer gcc and/or
linux distribution).

This is what evidently happens. A clang binary distribution gets
created for a particular Linux distro but there is no compatibility in
the package which states that it only works with some version of
libstdc++ or glibc or earlier. Then a later version of gcc gets
installed and with it a later version of libstdc++ and glibc as a
dependency. Now the binary version of clang no longer works on that
system because of the updates to the C/C++ standard library.

Then, the package system of your Linux distribution is broken. Or the
package containing the Clang binaries does not correctly state its
dependencies.

[snip]

Generally, it is recommended to bootstrap clang:

1. Build llvm/clang with system gcc
2. Build llvm/clang with new clang
3. Install new clang somewhere (which will automatically use system
libstdc++/glibc).

This should solve your problem.

[I think Justin's recipe is overkill, but whatever]

OK, this sounds good. Can this process not be explained on the clang
Getting Started page for those who are building clang from source. I
am not sure how step 3) is supposed to go. Do I just copy the built
bin directory somewhere else and then put it first in my PATH, before
running llvm/configure ?

You can install on a temporary directory or directly use the newly
created clang binaries, setting the environment variables CC and CXX
with full paths like this:

CC=/path/to/clang CXX=/path/to/clang++ configure [configure options] && make

Changing the PATH should work, but remeber to undo the change
afterwards.

Or is "Install new clang" more complicated than that ?

It is simply `sudo make install' (which installs on system-level default
directories.) You can also install on a custom directory, using the
parameter `--prefix=' in the `configure' invocation.

Originally I thought I could have both an earlier binary version of
clang as well as the latest built version of clang in a Linux distro
so that I could test code on both for compatibility of my library for
end-users. But with the fairly common breakage of the binary version
package of clang when libstdc++/glibc gets updated, this has become
much less of a reality.

See above. You can have as much Clang versions as you please. Just be
sure to not uninstall the required gcc version for each Clang version.

Edward Diener <eldlistmailingz@tropicsoft.com> writes:

>> Clang will attempt to use your installed system libraries, usually
>> libstdc++ and glibc on a Linux distribution. We could probably maintain
>> a libstdc++ compatibility table that says clang x.y is compatible with
>> libstdc++ up to z.w. In the vast majority of cases, this just works if
>> you install a somewhat-newer version of clang. The problems arise when
>> you use an older clang with a newer libstdc++ (e.g. newer gcc and/or
>> linux distribution).
>
> This is what evidently happens. A clang binary distribution gets
> created for a particular Linux distro but there is no compatibility in
> the package which states that it only works with some version of
> libstdc++ or glibc or earlier. Then a later version of gcc gets
> installed and with it a later version of libstdc++ and glibc as a
> dependency. Now the binary version of clang no longer works on that
> system because of the updates to the C/C++ standard library.

Then, the package system of your Linux distribution is broken. Or the
package containing the Clang binaries does not correctly state its
dependencies.

This should only really be a problem for rolling-release distributions like
Arch. For others, even if gcc is updated, the system libstdc++ probably
won't be for compatibility reasons.

[snip]

>> Generally, it is recommended to bootstrap clang:
>>
>> 1. Build llvm/clang with system gcc
>> 2. Build llvm/clang with new clang
>> 3. Install new clang somewhere (which will automatically use system
>> libstdc++/glibc).
>>
>> This should solve your problem.

[I think Justin's recipe is overkill, but whatever]

Sure, you can just build with system gcc and be done with it. Though being
a C++ program itself, building clang successful should give you some
reassurance that it will work for other C++ programs.

> OK, this sounds good. Can this process not be explained on the clang
> Getting Started page for those who are building clang from source. I
> am not sure how step 3) is supposed to go. Do I just copy the built
> bin directory somewhere else and then put it first in my PATH, before
> running llvm/configure ?

You can install on a temporary directory or directly use the newly
created clang binaries, setting the environment variables CC and CXX
with full paths like this:

CC=/path/to/clang CXX=/path/to/clang++ configure [configure options] &&
make

Changing the PATH should work, but remeber to undo the change
afterwards.

> Or is "Install new clang" more complicated than that ?

It is simply `sudo make install' (which installs on system-level default
directories.) You can also install on a custom directory, using the
parameter `--prefix=' in the `configure' invocation.

> Originally I thought I could have both an earlier binary version of
> clang as well as the latest built version of clang in a Linux distro
> so that I could test code on both for compatibility of my library for
> end-users. But with the fairly common breakage of the binary version
> package of clang when libstdc++/glibc gets updated, this has become
> much less of a reality.

See above. You can have as much Clang versions as you please. Just be
sure to not uninstall the required gcc version for each Clang version.

Does clang still maintain a hard-coded list of search-paths for GCC
versions?

Edward Diener <eldlistmailingz@tropicsoft.com> writes:

Clang will attempt to use your installed system libraries, usually
libstdc++ and glibc on a Linux distribution. We could probably maintain
a libstdc++ compatibility table that says clang x.y is compatible with
libstdc++ up to z.w. In the vast majority of cases, this just works if
you install a somewhat-newer version of clang. The problems arise when
you use an older clang with a newer libstdc++ (e.g. newer gcc and/or
linux distribution).

This is what evidently happens. A clang binary distribution gets
created for a particular Linux distro but there is no compatibility in
the package which states that it only works with some version of
libstdc++ or glibc or earlier. Then a later version of gcc gets
installed and with it a later version of libstdc++ and glibc as a
dependency. Now the binary version of clang no longer works on that
system because of the updates to the C/C++ standard library.

Then, the package system of your Linux distribution is broken. Or the
package containing the Clang binaries does not correctly state its
dependencies.

The package does state its dependencies and in the case I am now looking at, in Fedora 17, says that clang3.0 it was upgraded for compatibility with gcc4.7.2. When I tried to build llvm/clang itself using the clang3.0 package on Fedora 17 I received the error:

"In file included from /home/fceldiener/vcs/llvm/include/llvm/Support/SwapByteOrder.h:20:
/usr/bin/../lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/limits:1404:27: error: use of undeclared identifier
       '__int128'; did you mean '__int128_t'?
     struct numeric_limits<__int128>
                           ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/limits:1478:36: error: expected '>'
     struct numeric_limits<unsigned __int128>
                                    ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/limits:1478:5: error: cannot combine with previous '(error)'
       declaration specifier
     struct numeric_limits<unsigned __int128>
     ^
/usr/bin/../lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/limits:1478:44: error: expected unqualified-id
     struct numeric_limits<unsigned __int128>"

When I used gcc4.7.2 to build llvm/clang I had no problems.

On other Linux distros I had other errors. I will report each one here in separate threads. I had originally reported the error above in the clang user mailing list.

[snip]

Generally, it is recommended to bootstrap clang:

1. Build llvm/clang with system gcc
2. Build llvm/clang with new clang
3. Install new clang somewhere (which will automatically use system
libstdc++/glibc).

This should solve your problem.

[I think Justin's recipe is overkill, but whatever]

OK, this sounds good. Can this process not be explained on the clang
Getting Started page for those who are building clang from source. I
am not sure how step 3) is supposed to go. Do I just copy the built
bin directory somewhere else and then put it first in my PATH, before
running llvm/configure ?

You can install on a temporary directory or directly use the newly
created clang binaries, setting the environment variables CC and CXX
with full paths like this:

CC=/path/to/clang CXX=/path/to/clang++ configure [configure options] && make

Changing the PATH should work, but remeber to undo the change
afterwards.

Or is "Install new clang" more complicated than that ?

It is simply `sudo make install' (which installs on system-level default
directories.) You can also install on a custom directory, using the
parameter `--prefix=' in the `configure' invocation.

Thanks for this information