Deprecating autoconf/make?

Hi All,

I fear starting another centi-thread on this but I'll give it a shot.

We're currently supporting two build systems which is definitely one
more than we (or I) want to support.

I don't know of any support in autoconf past the
--host/--build/--target case that's not supported in cmake. I'll send
out an assertion here that this support isn't necessary and any system
that needs it can instead move to:

build native clang
build next clang with some target that supplies a sysroot and a
-target option to the native clang

Can anyone see good a reason not to move to cmake as our only build
configuration system and drop future support for autoconf + makefiles
now that 3.3 has branched?

-eric

I definitely think this is a good idea, and a very timely suggestion. Once we have collected a list of remaining problems, we should have plenty of time to address them all such that we can remove the autoconf build system for 3.4.

[Adding cfe-dev to CC, not everyone there is subscribed to llvmdev]

Hi All,

I fear starting another centi-thread on this but I'll give it a shot.

We're currently supporting two build systems which is definitely one
more than we (or I) want to support.

I don't know of any support in autoconf past the
--host/--build/--target case that's not supported in cmake. I'll send
out an assertion here that this support isn't necessary and any
system
that needs it can instead move to:

build native clang
build next clang with some target that supplies a sysroot and a
-target option to the native clang

Can anyone see good a reason not to move to cmake as our only build
configuration system and drop future support for autoconf + makefiles
now that 3.3 has branched?

We still need to add CMake support to the test suite, right?

-Hal

CMake is just a makefile (or <insert build system here>) generator. So
something like cmake <stuff>; make check-all works already.

Or did you have something else in mind?

-eric

CMake is just a makefile (or <insert build system here>) generator.
So
something like cmake <stuff>; make check-all works already.

Or did you have something else in mind?

I meant the stuff in projects/test-suite. Maybe CMake can just invoke the necessary sub-configure?

-Hal

CMake is just a makefile (or <insert build system here>) generator.
So
something like cmake <stuff>; make check-all works already.

Or did you have something else in mind?

I meant the stuff in projects/test-suite. Maybe CMake can just invoke the necessary sub-configure?

Can be done, and then we can hopefully migrate projects/test-suite to cmake.

-eric

I can think of several:

- When the Clang CMakefiles go to build compiler-rt (if present), they use the compiler used to build Clang and not the newly-built Clang itself.

This one is fairly minor (I think), but it does make CMake as it stands untenable for cross-compilation builds where the host and target are not the same--the compiler-rt libs get built for the *host* and not for the *target*. But as you pointed out, there is a workaround (albeit a clumsy workaround).

Just something to keep in mind if and when you do move to CMake.

- CMake is not available everywhere. Bourne shells are*.

As I recall, this is the reason autoconf generates shell scripts: because nearly every operating system out there comes with a shell! Even the old-timey MPW development environment for the Mac came with one! (* Even Windows comes with a shell, though it's not a Bourne shell but something else entirely with a few constructs borrowed from the Bourne shell's predecessor, the Thompson shell. But I only care about Windows inasmuch as people still use it anyway.)

In particular, CMake is not distributed with Xcode. This means that engineers at Apple--who may I remind you all still drives the bulk of LLVM, Clang, and especially LLDB development, though IBM, Intel, AMD, and especially Google have really stepped up their contributions recently--and for whom you yourself, Eric, once worked until somewhat recently--won't be able to build their own shiny new toolchain at all, unless they can somehow convince management to install CMake. In fact I seem to recall quite recently someone from Apple (Jim Ingham?) on one of the LLVM lists complaining that he wanted to use ninja, but couldn't because he couldn't get ninja and CMake installed on his workstation. Until this changes--and good luck with that--expect heavy resistance from anyone still at Apple (other than Doug Gregor, of course :).

- Some people around here don't like CMake very much, if at all--Daniel Dunbar, I'm looking at you!

I've brought this up before, but I distinctly recall Daniel saying he had some kind of plan for making a new Python-based build system for LLVM. Those LLVMBuild files scattered throughout the tree? That was supposed to be the start of it. I don't know if he's abandoned those plans or what, but I think we should consider all our options before we commit to CMake. (Then again, Python isn't exactly everywhere either. It's more widely distributed than CMake--Mac OS X and damn near every Linux distro comes with it--but again, Windows doesn't come with it either. And I recall some other people raising objections to a Python build system because of that...)

Heck, I'll admit that *I* don't like CMake that much, either. It's certainly better than "autohell" (as one person on this list put it), I'll give it that. But I think we can do better than CMake. Granted, CMake is the easiest alternative to autoconf, since we have most of the infrastructure for it in place already, but what will we do when sometime (5, 10 years?) down the line we find that CMake is inadequate and want to move off of it? This--the beginning of the next release cycle, as others have pointed out--is exactly the right time to consider alternative build systems so we can work the bugs out of them in time for 3.4.

Now if you'll excuse me, I need to go put on a flame-retardant suit...

Chip

Can anyone see good a reason not to move to cmake as our only build
configuration system and drop future support for autoconf + makefiles
now that 3.3 has branched?

I know of a small bug (pr14189) which breaks LTO and a bigger one:
when building compiler-rt the host compiler is used.

For these reasons I currently uses cmake for day to day development
but have to use configure to build the clang as I use as my system
compiler. If those issues are fixable, I would love to let autotools
go.

Cheers,
Rafael

I'm not sure if this is still the case, but at one point autoconf and
CMake were generating shared objects differently. autoconf builds placed
everything in one shared object, while cmake builds placed each component in
its own shared library.

I don't think this is something that would prevent LLVM from switching
to CMake, but it is one potential trouble spot for users.

-Tom

> Hi All,
>
> Can anyone see good a reason not to move to cmake as our only build
> configuration system and drop future support for autoconf + makefiles
> now that 3.3 has branched?

+1.

- CMake is not available everywhere. Bourne shells are*.

As I recall, this is the reason autoconf generates shell scripts: because
nearly every operating system out there comes with a shell! Even the
old-timey MPW development environment for the Mac came with one! (* Even
Windows comes with a shell, though it's not a Bourne shell but something
else entirely with a few constructs borrowed from the Bourne shell's
predecessor, the Thompson shell. But I only care about Windows inasmuch as
people still use it anyway.)

I just want to point out that that's a huge asterisk.

In particular, CMake is not distributed with Xcode. This means that

engineers at Apple--who may I remind you all still drives the bulk of LLVM,
Clang, and especially LLDB development, though IBM, Intel, AMD, and
especially Google have really stepped up their contributions recently--and
for whom you yourself, Eric, once worked until somewhat recently--won't be
able to build their own shiny new toolchain at all, unless they can somehow
convince management to install CMake. In fact I seem to recall quite
recently someone from Apple (Jim Ingham?) on one of the LLVM lists
complaining that he wanted to use ninja, but couldn't because he couldn't
get ninja and CMake installed on his workstation. Until this changes--and
good luck with that--expect heavy resistance from anyone still at Apple
(other than Doug Gregor, of course :).

Do you mean Jim Grosbach here?
http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-April/061330.html

My take away was that switching to CMake is a large but superficial
workflow change that takes time to get used to.

w.r.t. the comments about ninja, LLVM doesn't need to push that as the
preferred build system. 'make' works fine out of the box. ninja is just
useful if you want faster builds and are willing to build your own binary.

Here's an angle I haven't heard: CMake can generate XCode project files.
I'm not sure that matters to anyone who works on LLVM, but it would make
it easier for Apple to dogfood their own IDE. I also hear the generated
project files are not so good, though.

Yes please.

On the practical side, our internal CI does cmake/autoconf x debug/release
x gcc/clang (on linux). Cutting out autoconf reduces the number of
combinations per build from 8 to 4. This probably doesn't matter to most
people but it would be great for us.

paul

Paul, you can also just not test autoconf. Why does it matter to you if the autoconf files are there or not ?

Paul, you can also just not test autoconf. Why does it matter to you if the autoconf files are there or not ?

We want to make sure that any code we upstream is well tested. This includes building with both cmake and autoconf to ensure we don't break one build system or the other. That's the same reason the system builds with both gcc and clang. Most people here use cmake so our CI checks autoconf builds for them—that's the theory anyway :slight_smile:

paul

I'm not sure where you got the "cmake isn't included in xcode, therefore noone can use it on an apple platform". That is just utter nonsense.

BTW, whoever cross posted this to three different lists, please do *not* do this. It makes moderation a huge PITA.

-Chris

It is–in more ways than one :). Besides being pretty long, I’m well aware that there’s quite a few Windows users on this list. Many of them don’t have a shell that can run configure installed. For them, ditching autoconf in favor of CMake is a no-brainer. Why put so much effort into maintaining a build system a sizable proportion of our users can’t even use?

From the Unix point of view, Windows is very odd, isn’t it? But I digress.

Yeah, that’s who I meant.

Huh. My memory must be a bit fuzzy… I could swear I recall someone saying “My boss won’t let me install CMake on my workstation” or something to that effect.

I know.

True.

This is (IIRC) one of the reasons Daniel Dunbar doesn’t like CMake.

Chip

In particular, CMake is not distributed with Xcode. ... couldn't because he couldn't get ninja and CMake installed on his workstation.

I'm not sure where you got the "cmake isn't included in xcode, therefore noone can use it on an apple platform". That is just utter nonsense.

That is a *gross* misrepresentation of what I said.

Anyone can go install CMake themselves by downloading it from their website; I thought that was obvious! That's why I didn't say it. What I was specifically saying there was, some people who *work* at companies like Apple have bosses that may or may not let them install CMake, which might be a problem if they need to build LLVM and there's no longer a handy pre-fab shell script to generate makefiles with. Sorry if that wasn't clear.

Also, from what I can tell, Apple's internal build processes seem to be highly integrated at the moment with configure+make, which is yet another barrier to switching to pure CMake.

BTW, whoever cross posted this to three different lists, please do *not* do this. It makes moderation a huge PITA.

Sorry. I didn't know better. I've taken it off the other lists.

Chip

In particular, CMake is not distributed with Xcode. ... couldn't because he couldn't get ninja and CMake installed on his workstation.

I'm not sure where you got the "cmake isn't included in xcode, therefore noone can use it on an apple platform". That is just utter nonsense.

That is a *gross* misrepresentation of what I said.

Anyone can go install CMake themselves by downloading it from their website; I thought that was obvious! That's why I didn't say it.

Ok, I'm sorry I misunderstood you, but...

What I was specifically saying there was, some people who *work* at companies like Apple have bosses that may or may not let them install CMake,

I still don't get what you're saying here. I agree that, in theory, there may be some people who work at companies who don't let folks use Cmake, but Apple is certainly not one of them. Can you give a concrete reason that you think that this is the case, or why you think it might be the case for some other company?

Also, from what I can tell, Apple's internal build processes seem to be highly integrated at the moment with configure+make, which is yet another barrier to switching to pure CMake.

Just MHO, but instead of telling the world why you think that Apple's build process won't like this, why not let someone at apple speak up if they think it is a problem?

-Chris

Hi Eric! Does cmake support in-tree builds yet?

Nick

Just a bit of info in case this isn’t widely known and may be useful…

One can drive the MS compiler (cl.exe) and linker (link.exe) from Cygwin if he starts the cygwin shell from a Visual Studio command prompt which sets up the required environment variables. So, it is certainly feasible to use automate/autoconf on Windows if one chooses. The only irritation I ran into doing that was with path names. If memory serves me correctly, I had trouble passing absolute path names to things like the header search path switch. I had to use relative paths.

-James