Autotools support for building sanitizers was never complete, is a proper subset of CMake capabilities, and the gap between the features of CMake build and autotools is quickly increasing:
OS: autotools support Linux, Mac OS X and Android; CMake also supports FreeBSD and Windows.
Architectures: autotools support i386 and x86_64, CMake also supports MIPS, ARM/AArch64, PowerPC).
Library variants: autotools don’t support building shared ASan runtime on Linux.
4 [!!!]) Autotools build doesn’t have any support for building and running tests.
The sets of compiler flags we use to build runtimes are different in two builds, and are hard to kept in sync.
Because of (4), sanitizer runtimes built with autotools are severely undertested, and maintaining two different build systems is a burden I would like to get rid of. Now seems to be a good time for that: we’ve cut off 3.6 release branch, and Hans assures me that 3.7 release process will use CMake.
Are there existing users of configure/make that want sanitizers to be available in their build? Are there other arguments for keeping the autotools build “working” I’m missing?
From: "Chandler Carruth" <chandlerc@google.com>
To: "Alexey Samsonov" <vonosmas@gmail.com>
Cc: "Nico Weber" <nicolasweber@gmx.de>, "Clang Developers List" <cfe-dev@cs.uiuc.edu>, "LLVM Dev"
<llvmdev@cs.uiuc.edu>
Sent: Thursday, February 12, 2015 8:41:18 PM
Subject: Re: [LLVMdev] RFC: Dropping support for building sanitizers with autotools
Hi everyone,
TL;DR I plan to do subj unless anyone objects.
Autotools support for building sanitizers was never complete, is a
proper subset of CMake capabilities, and the gap between the
features of CMake build and autotools is quickly increasing:
1) OS: autotools support Linux, Mac OS X and Android; CMake also
supports FreeBSD and Windows.
2) Architectures: autotools support i386 and x86_64, CMake also
supports MIPS, ARM/AArch64, PowerPC).
3) Library variants: autotools don't support building shared ASan
runtime on Linux.
4 [!!!]) Autotools build doesn't have *any* support for building and
running tests.
5) The sets of compiler flags we use to build runtimes are different
in two builds, and are hard to kept in sync.
Because of (4), sanitizer runtimes built with autotools are severely
undertested, and maintaining two different build systems is a burden
I would like to get rid of. Now seems to be a good time for that:
we've cut off 3.6 release branch, and Hans assures me that 3.7
release process will use CMake.
+1, ship it.
I agree. I've found the autotools building support fairly useless precisely because there's no way to run the tests.
Sounds fine to me, but be aware that some of the libraries in compiler-rt other than the sanitizers currently are only built correctly in the autotools build (for example, I think libprofile_rt for iOS isn’t built correctly in the cmake build).
Sounds fine to me, but be aware that some of the libraries in compiler-rt
other than the sanitizers currently are only built correctly in the
autotools build (for example, I think libprofile_rt for iOS isn't built
correctly in the cmake build).
Sure, I don't plan to remove support for builtins and libprofile.
Unfortunately, we do rely on the Autotools to build for darwin.
The lack of proper testing is very unfortunate. In the future, we will either drop the dependency on Makefile/autoconf build or add the support for sanitizer testing.
Unfortunately, we do rely on the Autotools to build for darwin.
The lack of proper testing is very unfortunate. In the future, we will
either drop the dependency on Makefile/autoconf build or add the support
for sanitizer testing.
Sorry, for some reason I had the impression that you're also more
interested in CMake buiild :-/. Only now I noticed that recent Kuba's
patches about -isysroot fix both makefile and CMake
build system (and he's probably not very happy about it).
I believe the effort required to port sanitizer unit tests to autotools is
not worth it, and it's better to spend that time migrating the release
tools to CMake. Keeping two build systems "working"
was and is a manageable, but unpleasant burden. Providing a good test
coverage for two runtime versions is hard, and keeping two runtime versions
"equivalent" (built with the same compiler flags)
is close to impossible.
I understand that removing autotools support right now breaks your
integration. What if we just delete autotools support from all Linux builds?
Speaking of autotools support, I should lay out the reason why I raised the
question of removing it last week. We plan to make intrusive changes to
ASan and UBSan library layout:
1) embed UBSan runtime into ASan to solve various initialization and
integration problems of using ASan+UBSan (fixing http://llvm.org/bugs/show_bug.cgi?id=21112)
2) make UBSan runtime standalone, and forbid to use it with another
sanitizers except for ASan.
Unfortunately, it means changing a bunch of build rules in *both* CMake and
makefile build.
Ideally, I would be happy to transfer ownership of current autotools build
on Darwin to you, and ask for help in implementing/testing necessary
changes on Mac OS X side
At the moment we don't even have a live buildbot running autotools-built
ASan on Mac, so you're in much better shape to catch and notice regressions.
Unfortunately, we do rely on the Autotools to build for darwin.
The lack of proper testing is very unfortunate.
This isn't just "proper testing". This is any testing at all. I have to
guess that you're not using it for *development*, just using it as part of
some release process build system?
If that's the case, I think it would be better for you to maintain whatever
you need to support that in the interim out-of-tree. If the tests don't
run, I don't think it's really reasonable to keep a build system in tree
and expect open source contributors to update it and keep it working. How
will they even know if it is working? We don't even have build bots that
cover it as far as I know? (If we do, that somewhat lessens this pain...)
In the future, we will either drop the dependency on Makefile/autoconf
build or add the support for sanitizer testing.
Given the push to move away from autotools altogether, it would seem really
weird to add support for testing to it rather than migrating the dependency
away. You can even use the cmake build *from* an autoconf/Makefile setup by
running cmake directly. It's a bit annoying, but it can be done.
There are still a lot of issues blocking full CMake equivalence with autoconf/Makefile (PR15732). Removing the ASan makefile support is really premature.
Unfortunately, we do rely on the Autotools to build for darwin.
The lack of proper testing is very unfortunate. In the future, we will either drop the dependency on Makefile/autoconf build or add the support for sanitizer testing.
Sorry, for some reason I had the impression that you’re also more interested in CMake buiild :-/. Only now I noticed that recent Kuba’s patches about -isysroot fix both makefile and CMake
build system (and he’s probably not very happy about it).
I believe the effort required to port sanitizer unit tests to autotools is not worth it, and it’s better to spend that time migrating the release tools to CMake. Keeping two build systems “working”
was and is a manageable, but unpleasant burden. Providing a good test coverage for two runtime versions is hard, and keeping two runtime versions “equivalent” (built with the same compiler flags)
is close to impossible.
I agree. The solution where we only rely on cmake build would be the best option.
I understand that removing autotools support right now breaks your integration. What if we just delete autotools support from all Linux builds?
Yes. That is fine by me.
Speaking of autotools support, I should lay out the reason why I raised the question of removing it last week. We plan to make intrusive changes to ASan and UBSan library layout:
make UBSan runtime standalone, and forbid to use it with another sanitizers except for ASan.
Unfortunately, it means changing a bunch of build rules in both CMake and makefile build.
Ideally, I would be happy to transfer ownership of current autotools build on Darwin to you, and ask for help in implementing/testing necessary changes on Mac OS X side
At the moment we don’t even have a live buildbot running autotools-built ASan on Mac, so you’re in much better shape to catch and notice regressions.
We do have public buildbots set up for OS X:
Here is one that is testing a makefile build. There are probably others in different configurations.
I may be missing something obvious (the jenkins UI is very unfamiliar to
me) but I don't see this build bot failing at any time because of this
change... Is it actually building the ASan runtimes?
The change has been reverted shortly after the initial commit to allow for more discussions. This is why the bot is green. You can see that the asan library is being built if you choose “Console Output” menu and click “full log”, search for “libclang_rt.asan_osx_dynamic.dylib”.
I don’t think there is a specific reason why we need the sanitizers to be built with autotools. I agree that the current setup is broken in several ways and needs attention. Thank you for the suggestion to run cmake from an autoconf/Makefile setting; we’ll investigate if that could be an acceptable solution.
Unfortunately, we do rely on the Autotools to build for darwin.
The lack of proper testing is very unfortunate. In the future, we will
either drop the dependency on Makefile/autoconf build or add the support
for sanitizer testing.
Sorry, for some reason I had the impression that you're also more
interested in CMake buiild :-/. Only now I noticed that recent Kuba's
patches about -isysroot fix both makefile and CMake
build system (and he's probably not very happy about it).
I believe the effort required to port sanitizer unit tests to autotools is
not worth it, and it's better to spend that time migrating the release
tools to CMake. Keeping two build systems "working"
was and is a manageable, but unpleasant burden. Providing a good test
coverage for two runtime versions is hard, and keeping two runtime versions
"equivalent" (built with the same compiler flags)
is close to impossible.
I agree. The solution where we only rely on cmake build would be the best
option.
I understand that removing autotools support right now breaks your
integration. What if we just delete autotools support from all Linux builds?
Yes. That is fine by me.
OK, I've removed the support for sanitizers in makefile/autoconf builds on
Linux in r229754-r229756.
Can you point at which open bugs in that list actually affect packaging for
Debian? I assume http://llvm.org/pr15493, which is about building a
monolithic libLLVM-N.M.so, is the main one and nothing else matters.
When packaging, do you build all the LLVM subprojects together or
separately one after another? If you build separately, you should be able
to build compiler-rt with cmake and keep building LLVM with autoconf.
