Is there room for another build system?

Project Infrastructure LLVM Dev List Archives
1

[Repost: The mailing list was down yesterday. Sorry if this is a
duplicate]

I'm evaluating CMake (1) primarily as an alternative build system for
Visual Studio users, although it can easily be a replacement for
`configure' and hand-made makefiles too, providing a single build system
for all platforms. CMake is a tool that takes a project description and
configures, generates makefiles, project files for IDEs, etc as
requested.

There are three MS C++ compilers capable of compiling LLVM: Visual
Studio 2003, 2005, 2008 and their respective "free tools"
edition. However, the project files distributed with LLVM are for
2005. This means that people using 2003 is left in the cold and people
who only have 2008 must convert the project files and, most important,
can't contribute to the maintenance of the VC++ build.

CMake can generate Visual Studio project files (for all three versions)
and NMake makefiles. This solves the schism among Visual Studio users.

Furthermore, CMake can replace `configure' and generate gmake makefiles,
project files for XCode, etc.

In just an afternoon I hacked CMake specifications for the LLVM
libraries and successfully built the fibonacci example. Most of the time
consisted on just waiting for the build to complete on each
experimentation cycle, so I stimate that was less that 2 hours of real
activity. This without previous knowledge of CMake (nor of the VC++
build). So my impression is quite favorable.

CMake just requires one plain text file named CMakeLists.txt on every
source directory, and the only maintenance is to keep up to date the
list of source files of the directory. So, the work it causes is no more
than the current VC++ build. Anyone, even those who doesn't have the
Visual Studio IDE, can do it. It's pretty trivial.

Now, my questions:

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are interested,
I can steer my work to cover all platforms.

2. VC++ users: Are you interested on a tool that generates project
files or makefiles for your IDE or your free VC++ compiler, and with a
simpler maintenance?

3. LLVM administrators: Do you object to having a CMakeLists.txt file
on every source directory?

Obviously, if the answer to 3 is `yes', previous questions are
irrelevant :slight_smile:

(1): http://www.cmake.org

2

Hi Oscar,

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are interested,
I can steer my work to cover all platforms.

We (RapidMind) are very interested. We would very much like to see a unified build system across MSVC/Windows and gcc/Linux/OS X.

We have considered contributing such a build system, and if we were to do so would probably base it on SCons (http://www.scons.org/) because we already use SCons extensively. I have a small prototype of this. We're not really partial to a particular implementation though, as long as it allows a unified, scriptable, build environment across the various platforms. SCons has the nice property that its build scripts are Python, so you can do just about anything with it.

2. VC++ users: Are you interested on a tool that generates project
files or makefiles for your IDE or your free VC++ compiler, and with a
simpler maintenance?

Yes :slight_smile:

Are you intending for your build system to support all the features of the existing make-based build system, e.g. running the test suite, supporting cross compilation, etc.?

3

I'm evaluating CMake (1) primarily as an alternative build system for
Visual Studio users, although it can easily be a replacement for
`configure' and hand-made makefiles too, providing a single build system
for all platforms. CMake is a tool that takes a project description and
configures, generates makefiles, project files for IDEs, etc as
requested.

Ok. Killing off autoconf would be a huge bonus, but should probably be done as a second step.

There are three MS C++ compilers capable of compiling LLVM: Visual
Studio 2003, 2005, 2008 and their respective "free tools"
edition. However, the project files distributed with LLVM are for
2005. This means that people using 2003 is left in the cold and people
who only have 2008 must convert the project files and, most important,
can't contribute to the maintenance of the VC++ build.

Yes, this is a problem. We also support Xcode to various degrees on the mac, and keeping all these project files in sync with the makefiles is a pain.

Furthermore, CMake can replace `configure' and generate gmake makefiles,
project files for XCode, etc.

Having one point of truth would be great.

Now, my questions:

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are interested,
I can steer my work to cover all platforms.

I would prefer to kill off makefiles if we have something better. We really only want to support one build system.

3. LLVM administrators: Do you object to having a CMakeLists.txt file
on every source directory?

No, particularly if it means we eventually lose the Makefile :).

One trick is that we want to keep llvm-config and the 'LINK_COMPONENTS' system used when building tools. I am sure cmake can handle this, but I just wanted to mention that it is important.

-Chris

4

Chris Lattner <clattner@apple.com> writes:

Ok. Killing off autoconf would be a huge bonus, but should probably
be done as a second step.

My plan is a staged one:

First, support MSVC++.

Second, implement what `configure' does now. MSVC++ users would be the
first ones to take advantage of this, instead of the current hack Visual
Studio does.

Finally, add capabilities for replacing current hand-made makefiles on
all platforms.

Once the work is done, killing off autoconf and hand-made makefiles is
up to you. CMake can live with them and the amount of maintenance work
it adds is minimal.

[snip]

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are
interested,
I can steer my work to cover all platforms.

I would prefer to kill off makefiles if we have something better. We
really only want to support one build system.

My knowledge of the LLVM build system is far from complete, but right
now there is one feature which I think is tricky to implement on CMake:
if you add/remove a source file, the build system we have now does the
right thing. In CMake, you are required to update the list of source
files on the CMakeLists.txt file. Actually, it is possible for CMake to
read the contents of the source directory on the configure phase, but
this disallows other interesting things.

Other tricky thing is to integrate user code on the LLVM build
system. I'm thinking on people who is coding their own extensions to
LLVM, for instance, and that work outside the LLVM source tree.

I need to read again the docs about the LLVM build system.

3. LLVM administrators: Do you object to having a CMakeLists.txt file
on every source directory?

No, particularly if it means we eventually lose the Makefile :).

Okay, I'll keep working on this project then.

One trick is that we want to keep llvm-config and the
'LINK_COMPONENTS' system used when building tools. I am sure cmake
can handle this, but I just wanted to mention that it is important.

Actually, I plan to use llvm-config (and LINK_COMPONENTS) even for VC++
users :-). Probably as an alternative, as some Visual Studio users would
complain if Perl is a requirement for building LLVM.

5

Chris Lattner <clattner@apple.com> writes:

Ok. Killing off autoconf would be a huge bonus, but should probably
be done as a second step.

My plan is a staged one:

First, support MSVC++.

Second, implement what `configure' does now. MSVC++ users would be the
first ones to take advantage of this, instead of the current hack Visual
Studio does.

Finally, add capabilities for replacing current hand-made makefiles on
all platforms.

Once the work is done, killing off autoconf and hand-made makefiles is
up to you. CMake can live with them and the amount of maintenance work
it adds is minimal.

Ok, this sounds like a great plan. I'm mostly aiming for this to be done in a series of incremental steps, rather than as a big bang :). Your plan sounds great!

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are
interested,
I can steer my work to cover all platforms.

I would prefer to kill off makefiles if we have something better. We
really only want to support one build system.

My knowledge of the LLVM build system is far from complete, but right
now there is one feature which I think is tricky to implement on CMake:
if you add/remove a source file, the build system we have now does the
right thing. In CMake, you are required to update the list of source
files on the CMakeLists.txt file.

I don't have a problem with this, this is acceptable.

One trick is that we want to keep llvm-config and the

'LINK_COMPONENTS' system used when building tools. I am sure cmake
can handle this, but I just wanted to mention that it is important.

Actually, I plan to use llvm-config (and LINK_COMPONENTS) even for VC++
users :-). Probably as an alternative, as some Visual Studio users would
complain if Perl is a requirement for building LLVM.

Cool! Thanks for working on this!

-Chris

6

Stefanus Du Toit <sdt@rapidmind.com> writes:

[snip]

We have considered contributing such a build system, and if we were to
do so would probably base it on SCons (http://www.scons.org/) because
we already use SCons extensively.

At first, SCons is what I intended too. But then I read about the KDE
experience and took the safe route :slight_smile:

[snip]

Are you intending for your build system to support all the features of
the existing make-based build system, e.g. running the test suite,
supporting cross compilation, etc.?

My initial plan was: if the LLVM devs are interested, go for the whole
pie; if just VC++ people is interested, restrict to VC++. Now that Chris
showed interest, I'll aim at fully replacing the current
system. However, as I'm just learning CMake and the LLVM build system,
things will come as my technical possibilities allows.

Thanks for your feedback.

7

Hi,

CMake just requires one plain text file named CMakeLists.txt on every
source directory, and the only maintenance is to keep up to date the
list of source files of the directory.

can it be kept up-to-date automagically? After all, you can query
subversion to get the list of all files in a directory.

1. General LLVM users: Are you so happy with `configure' and hand-made
makefiles that you wont consider an alternative? If you are interested,
I can steer my work to cover all platforms.

Do ordinary users need to have cmake if they want to build llvm?
If so, that's bad because they'll have to install it (unlike the
current setup, where only very standard tools are needed).

Ciao,

Duncan.

8

I too have done some work on an alternative build system for llvm. It is based on boost.build. Of course, it's quite a large project and I'm not ready to contribute the work yet. Ideally I was hoping to replace all of makefile functionality with jamfiles. Boost.build is attractive because support for new compilers/tools comes with new releases of boost.build and need not be added manually by llvm developers/users.

Since you're working on a competing build system, I'm no longer sure I want to spend time developing my build system. It would help to know what are your estimates for the availability timeframe of your build system.

Thanks,
Dmitri

9

Duncan Sands <baldrick@free.fr> writes:

Hi,

CMake just requires one plain text file named CMakeLists.txt on every
source directory, and the only maintenance is to keep up to date the
list of source files of the directory.

can it be kept up-to-date automagically? After all, you can query
subversion to get the list of all files in a directory.

Yes, it can, but CMake would do this at configure time, which means that
you'll need to explicitly invoke CMake. If you alter the CMakeLists.txt
file to reflect your change, next time you invoke your build (gmake,
nmake, Visual Studio...) it automatically notices this and does the
right thing.

Please note that CMake (unlike SCons) is not a replacement for `make',
it is a makefile generator, so the amount of intelligence it can inject
on its output (makefiles, Visual Studio projects, etc) is limited.

Do ordinary users need to have cmake if they want to build llvm?

Yes.

If so, that's bad because they'll have to install it (unlike the
current setup, where only very standard tools are needed).

That's something to count on the `against' side, yes.

10

"Makarov, Dmitri" <Dmitri.Makarov@amd.com> writes:

I too have done some work on an alternative build system for llvm. It
is based on boost.build. Of course, it's quite a large project and
I'm not ready to contribute the work yet. Ideally I was hoping to
replace all of makefile functionality with jamfiles. Boost.build is
attractive because support for new compilers/tools comes with new
releases of boost.build and need not be added manually by llvm
developers/users.

I tried boost.build on the past for other project and couldn't make head
or tails of it. Even the abstraction of compiler options it does does
not appeal too much to me. OTOH, can it replace autoconf?

I don't want to bash boost.build, just saying that it is not for me (as
a build system maintainer).

Since you're working on a competing build system, I'm no longer sure I
want to spend time developing my build system. It would help to know
what are your estimates for the availability timeframe of your build
system.

Well, the basic VC++ support is practically done. I want to enhance it
with llvm-config functionality. I hope to do this on a few days. I
stimate just a few hours of real work. The only real slow-down I see is
the long waits for every trial run on my old computer.

11

@Duncan Sands

I've worked with CMake for creating plugins for ParaView on MacOSX 10.4, 10.5, and Windows XP under Visual Studio 2005. I've also built other utilities on MacOSX 10.4, Cygwin and MSYS on WinXP, 32-bit and 64-bit Linux, and AmigaOS 4.0 using the traditional autotools.

CMake has an intuitive GUI for all of the user-defined functions so that, if a library cannot be found, a double-click on the entry and a click on a button pops up a file-requester so that the user can locate the library or directory and CMake also adjusts accordingly. If you only have a text-based terminal, a character-mapped GUI will be used instead.

The Autotools approach are specific to Posix-related operating systems and are not very portable to systems like Windows and the Amiga (which does not support fork() and other Posix-related functions) and I must say that I do not think that Autotools are ready for a cross-platform environment and certainly not an asset to LLVM.

In my not-so-humble opinion, Autotools should be packaged only with GCC and not tied to any other compilers such as LLVM's Clang project. Autotools are only designed to work with the GNU environment. CMake is designed to be cross-platform and even works within cross-development environments as of verision 2.6 .

My only complaint about CMake is that the XCode project generator support was buggy in version 2.6.0 but this should be fixed in 2.6.1 and so I may soon be able to put the Make utility to rest on the Mac in favor of a more native build environment.

I look forward to being able to use LLVM on many operating systems and not just Posix ones. I was even contemplating trying to port LLVM to AROS and, if the environment is right, I should be able to do it.

I'll be watching the list to see what develops. Maybe I'll even get more involved later on.

Sincerely,

Sam Crow

12

Duncan Sands wrote:

Do ordinary users need to have cmake if they want to build llvm?
If so, that's bad because they'll have to install it (unlike the
current setup, where only very standard tools are needed).

That's not the only problem with cmake. The autotools may be a big and ugly beast, but that's because they're trying to solve a big and ugly problem for which there is no silver bullet. And they are still much more comprehensive than cmake. I've considered cmake time and again for my own projects, but I don't think that it's quite there yet. Here are some points worth considering: Remlab: Issues with CMake (Some of these may already be addressed in newer cmake versions, I haven't checked recently.)

Albert

13

2. VC++ users: Are you interested on a tool that generates project
files or makefiles for your IDE or your free VC++ compiler, and with a
simpler maintenance?

Yes!

Already with a feature request - can it test for the required tools (bison/flex) are present and usable?

Cheers
George

14

I don't think flex is required. Bison is currently needed for one file (llvmAsmParser.y) AFAIK, which really needs to be rewritten anyway.

-Chris

15

George Russell <grrussel@googlemail.com> writes:

Already with a feature request - can it test for the required tools
(bison/flex) are present and usable?

Yes, at configure time.

Thanks for your feedback.

16

Albert Graef <Dr.Graef@t-online.de> writes:

[snip]

Here are some points worth considering:
Remlab: Issues with CMake (Some of these may already be
addressed in newer cmake versions, I haven't checked recently.)

Albert,

Some points you mention on your web page are solved. Others are not
applicable to LLVM. Others can be fixed within CMake itself (with some
effort, of course). I lack the knowledge to judge the points you raise
about distribution, package creation, etc. Finally, IMHO your concers
about CMake not preserving as many [autotools] feautures as [it] can,
does not apply to LLVM, at least as there is someone who vounteers to do
the replication work :slight_smile:

Please, some LLVM release manager (Tanya?), read Albert's web page and
evaluate how much impact have the issues he raises on your work.

17

My knowledge of the LLVM build system is far from complete, but right
now there is one feature which I think is tricky to implement on CMake:
if you add/remove a source file, the build system we have now does the
right thing. In CMake, you are required to update the list of source
files on the CMakeLists.txt file. Actually, it is possible for CMake to
read the contents of the source directory on the configure phase, but
this disallows other interesting things.

I solved that in linux some time ago with little external macros or
bash scripts, I think.

I did little more than just executing "ls" or "dir" and processing its
output as an string array.

alvaro

18

I've been using CMake already with LLVM for some time. I've managed to
build most of the libraries, though there are some issues left and I
haven't tested the things I don't use. I posted about it on this list
twice already but I didn't get any response.

The latest one is here:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2008-June/015541.html

It has a patch attached, so you can use the code if it's useful.

Regards,
Kevin André

19

Hi Kevin.

HyperQuantum <hyperquantum@gmail.com> writes:

I've been using CMake already with LLVM for some time. I've managed to
build most of the libraries, though there are some issues left and I
haven't tested the things I don't use. I posted about it on this list
twice already but I didn't get any response.

The latest one is here:
http://lists.cs.uiuc.edu/pipermail/llvmdev/2008-June/015541.html

It has a patch attached, so you can use the code if it's useful.

I found your work yesterday while searching the archives, and planned to
contact you once started with the non-VC++ platforms. Your work will be
very useful. Let's see if I solve the problems you found.

Thanks!

20

Óscar Fuentes wrote:

Albert Graef <Dr.Graef@t-online.de> writes:

Here are some points worth considering:
Remlab: Issues with CMake (Some of these may already be
addressed in newer cmake versions, I haven't checked recently.)

[...]

Please, some LLVM release manager (Tanya?), read Albert's web page and
evaluate how much impact have the issues he raises on your work.

Note that the URL I referred to is not mine. I merely wanted to point out that there are some possible issues to consider before throwing the existing build system out of the window. Which might affect LLVM users for whom the current build system works fine, like me. :slight_smile: