RFC: A proposal to move toward using C++11 features in LLVM & Clang / bounding support for old host compilers

Greetings,

This has been discussed many times, and there are a lot of pro’s and con’s on each side, but increasingly I think the project needs to draw a line in the sand and put in place long-term policies around support for building LLVM & Clang with older C++ toolchains. It also would have the benefit of removing divergence between LLVM sub-projects already using C++11 features. As they have grown in popularity and as code moves back and forth across project boundaries, I think this is a growing strain.

I am seriously concerned that if we don’t do this now, we will miss out on the accelerating trend of C++ as a language.

Concrete long term proposal:

We support building with C++ toolchains which were released and widely available on their respective target platforms at least 2 years prior to the next 6-month-cycle release. Immediately after each branch of a release, mainline can move the bar forward by 6 months. Thus, each release should build with the four previous releases.

Some observations:

  • I specifically am wording this to enable the mainline to use features sooner. People who are unable to use an even more modern toolchain will necessarily stay with each release.

  • This is an aggressive timeline. That is intentional. There is increasing pressure for LLVM to make use of new C++ language and library features.

  • There is always the option of checking out an old release, building it, and then using that compiler to build the latest release or to build the mainline compiler.

  • The goal is to only change the supported toolchain set and thus C++ dialect after a release branches to avoid confusion, churn, and impeding QA on a release. People can check out trunk and get an idea of whether they will have problems with the next release.

  • I have historically been more conservative on this topic, but listening to many people and looking at some of the C++ features we are missing, I think the right strategy for the project is a more aggressive one.

One short term caveat: Windows is special.

I don’t think we should follow the 2-year rule for Windows. It really is special for at least three reasons:

  1. We can’t bootstrap on Windows. While there is tons of exciting work going on here to change that, even then the quality of Windows support is likely to change very rapidly.
  2. Visual Studio’s C++ support moved very slowly for a long time, and is now improving at an amazing rate. We shouldn’t be hampered by past problems and should take advantage of this recent progress.
  3. We have less insight into when new versions of the MSVC++ toolchain will arrive.

So my suggestion is that, at least during the rapid evolution of VS’s C++11 support and Clang’s Windows support, we support building with the most recent 2 versions of VS available at the prior release. For example when we branch for 3.4, the two versions will be 2012 and 2013. Those two would be supported on mainline from then through 3.5, and when we branch for 3.5 we would re-evaluate.

NOTE: This would drop support for VS 2010!!! That is a huge change, hence my suggesting it only after we branch for 3.4. It gives us an entire release cycle of notice for people relying on it.

Now for the carrot: if we go with this plan, then immediately after branching for 3.4, we would be able to use the vast majority of C++11 features, targeting the following as the oldest toolchains supported through the 3.5 release timeframe:

GCC 4.7
Clang 3.1
VS 2012

Some notable features we would get to use:

  • r-value references, move semantics, etc
  • auto
  • range for loops
  • lambdas
  • static_assert
  • nullptr
  • std::unique_ptr, std::tuple, some other nice library stuff

These days, this list seems increasingly worth the cost of forcing users to get a modern toolchain onto their systems.

-Chandler

(re-sending to the actual mailing lists… go go gadget typos!)

GCC 4.7
Clang 3.1
VS 2012

Xcode 4.2's clang?

Do we already have bots with all these versions?

Cheers,
Rafael

Greetings,

This has been discussed many times, and there are a lot of pro's and con's on each side, but increasingly I think the project needs to draw a line in the sand and put in place long-term policies around support for building LLVM & Clang with older C++ toolchains. It also would have the benefit of removing divergence between LLVM sub-projects already using C++11 features. As they have grown in popularity and as code moves back and forth across project boundaries, I think this is a growing strain.

I am seriously concerned that if we don't do this now, we will miss out on the accelerating trend of C++ as a language.

Concrete long term proposal:

We support building with C++ toolchains which were released and widely available on their respective target platforms at least 2 years prior to the next 6-month-cycle release. Immediately after each branch of a release, mainline can move the bar forward by 6 months. Thus, each release should build with the four previous releases.

Some observations:
- I specifically am wording this to enable the mainline to use features sooner. People who are unable to use an *even more* modern toolchain will necessarily stay with each release.
- This is an aggressive timeline. That is intentional. There is increasing pressure for LLVM to make use of new C++ language and library features.
- There is always the option of checking out an old release, building it, and then using that compiler to build the latest release or to build the mainline compiler.
- The goal is to only change the supported toolchain set and thus C++ dialect after a release branches to avoid confusion, churn, and impeding QA on a release. People can check out trunk and get an idea of whether they will have problems with the next release.
- I have historically been more conservative on this topic, but listening to many people and looking at some of the C++ features we are missing, I think the right strategy for the project is a more aggressive one.

One short term caveat: Windows is special.

I don't think we should follow the 2-year rule for Windows. It really is special for at least three reasons:
1) We can't bootstrap on Windows. While there is tons of exciting work going on here to change that, even then the quality of Windows support is likely to change very rapidly.
2) Visual Studio's C++ support moved very slowly for a long time, and is now improving at an amazing rate. We shouldn't be hampered by past problems and should take advantage of this recent progress.
3) We have less insight into when new versions of the MSVC++ toolchain will arrive.

So my suggestion is that, at least during the rapid evolution of VS's C++11 support and Clang's Windows support, we support building with the most recent 2 versions of VS available at the *prior* release. For example when we branch for 3.4, the two versions will be 2012 and 2013. Those two would be supported on mainline from then through 3.5, and when we branch for 3.5 we would re-evaluate.

NOTE: This would drop support for VS 2010!!! That is a huge change, hence my suggesting it only *after* we branch for 3.4. It gives us an entire release cycle of notice for people relying on it.

Now for the carrot: if we go with this plan, then immediately after branching for 3.4, we would be able to use the vast majority of C++11 features, targeting the following as the oldest toolchains supported through the 3.5 release timeframe:

GCC 4.7
Clang 3.1
VS 2012

I do think that the time has come to switch to C++11 now as toolchains are becoming available everywhere. However, in my opinion a rule based on the releases of compilers won't work out for the majority of users out there. I'd rather base it on availability on operating systems, that's of course a fuzzy rule but it turns out that the number of common offenders shipping outdated toolchains is small.

My proposal is: look at the versions of OS's released in the last two years and check if they have shipped an outdated toolchain.

- Freebsd 10 is on Clang/libc++ by default, FreeBSD 9 also shipped Clang. Don't see problems here.
- Not sure about other BSD's, but I'd expect them to either follow FreeBSD or provide a newer GCC already.
- OS X is on Clang/libc++ by default now and has shipped Clang for a while. No issue here.
- Windows is special as people have to manage their own toolchain. However, we have aggressively dropped old MSVC versions in the past, I doubt dropping MSVC 2010 is a big problem as there's still 2012 and 2013 for people to pick from.

Then there's the long list of linux distros. I looked at those supported by Clang's toolchain config right now.
- Debian has (finally) switched to GCC 4.7. I don't think supporting old stable (Debian 6 is from 2011, GCC 4.4) really helps anyone.
- RHEL/CentOS is still on GCC 4.4 as far as I know. This is a problem if not solved within the next 6 months.
- SLES 11 was upgraded to GCC 4.7 in a service pack recently.
- Ubuntu is on GCC 4.7; LTS is on 4.6 but the next LTS release is likely out before LLVM 3.5 ships.
- The remaining distros have shorter release cycles (Arch, Exherbo, Fedora, OpenSUSE) and have migrated to a recent toolchain long ago.

So the only real problem is RHEL and its derivatives. Does anyone know if there's a new release imminent or an easy way to get a new toolchain on an existing installation?

- Ben

(Small nit and what it means for the conversation I don't know)
I *just* downloaded and installed Ubuntu (non-LTS) 1-2 weeks ago - I could be mistaken, but gcc out-of-the-box was 4.4 and fully updated it's 4.8.1

Interesting, I thought they only did compiler upgrades on major releases. But installing updates is a rather small obstacle compared to upgrading your whole system.

- Ben

Note that this is only for x86 and ARM. For PowerPC64 and MIPS, which look as if they'll be supported by LLVM/Clang soon (we now have PowerPC64 working, but the kernel must be compiled at -O0 or it doesn't boot, MIPS is still quite buggy, but getting there), we're still shipping gcc 4.2.1. These are tier 2 platforms, so we could cross-compile the toolchain from x86 for bootstrapping, but it would be quite painful.

David

- RHEL/CentOS is still on GCC 4.4 as far as I know. This is a problem if not solved within the next 6 months.

As far as I can see, there is "Red Hat Developer Toolset" which includes 4.7.2

I thought I'd throw in my two Windows cents...

- Windows is special as people have to manage their own toolchain. However, we have aggressively dropped old MSVC versions in the past,
I doubt dropping MSVC 2010 is a big problem as there's still 2012 and 2013 for people to pick from.

I guess the problem for me is _which_ MSVC to focus on. They release
updates regularly (seems to be a 3-6 month cycle), and Visual Studio
2012 as a whole is at Update 4 by now, I think.

I'm not sure what an update entails, but on the .NET side we've had
API breakage between updates. I don't know if the C++ folks release
major changes (e.g. changes to the compiler) at the same rate.

The "problem" with these incremental releases is that Visual Studio
insists that you update on every launch, so it's hard to know which
version you're really running locally unless you turn off automatic
updates. I haven't found a way to uninstall an update.

If the assumption is that the product gets progressively better, it
should be fine, but there's a risk that a sleight-of-hand click of an
Update button renders your machine LLVM/Clang-dev-incompatible.

FWIW,
- Kim

Benjamin Kramer <benny.kra@gmail.com>
writes:

- Ubuntu is on GCC 4.7; LTS is on 4.6

(Small nit and what it means for the conversation I don't know)
I *just* downloaded and installed Ubuntu (non-LTS) 1-2 weeks ago - I could be mistaken, but gcc out-of-the-box was 4.4 and fully updated it's 4.8.1

Interesting, I thought they only did compiler upgrades on major
releases. But installing updates is a rather small obstacle compared
to upgrading your whole system.

I just upgraded to Kubuntu 13.10. For 13.4, gcc was 4.7.? and now is
4.8.1. gcc 4.4 (among other versions) is available on the packages
repository, but not installed by default. The system compiler is
definitely gcc 4.8.1.

Greetings,

This has been discussed many times, and there are a lot of pro's and con's on each side, but increasingly I think the project needs to draw a line in the sand and put in place long-term policies around support for building LLVM & Clang with older C++ toolchains. It also would have the benefit of removing divergence between LLVM sub-projects already using C++11 features. As they have grown in popularity and as code moves back and forth across project boundaries, I think this is a growing strain.

Awesome, thank you for spearheading this discussion. We really need to move to some C++'11 support, it is just a question of how and when. We also had a discussion from a few months ago about this, can you summarize where we left off? I thought there was a tentative plan for LLVM 3.4 (or maybe it was after 3.4 branched).

I am seriously concerned that if we don't do this now, we will miss out on the accelerating trend of C++ as a language.

I'm not worried about the "accelerating trend", I just think we'll all be more productive.

Concrete long term proposal:

We support building with C++ toolchains which were released and widely available on their respective target platforms at least 2 years prior to the next 6-month-cycle release. Immediately after each branch of a release, mainline can move the bar forward by 6 months. Thus, each release should build with the four previous releases.

I don't think it makes sense to have a hard policy like this. It is fine to have a goal that allows us to continue picking up new features over time, but I think each such change will have to be discussed (at length) with the specifics of which interesting host configurations we would be dropping.

Some observations:
- I specifically am wording this to enable the mainline to use features sooner. People who are unable to use an *even more* modern toolchain will necessarily stay with each release.
- This is an aggressive timeline. That is intentional. There is increasing pressure for LLVM to make use of new C++ language and library features.
- There is always the option of checking out an old release, building it, and then using that compiler to build the latest release or to build the mainline compiler.
- The goal is to only change the supported toolchain set and thus C++ dialect after a release branches to avoid confusion, churn, and impeding QA on a release. People can check out trunk and get an idea of whether they will have problems with the next release.
- I have historically been more conservative on this topic, but listening to many people and looking at some of the C++ features we are missing, I think the right strategy for the project is a more aggressive one.

I really want this to happen, but it needs to be done in the right way, balancing the cost to the community to obsolete old configurations. As such, I agree with everything except your last point here :slight_smile:

One short term caveat: Windows is special.

I don't see how it is special. We should just look at all of the compilers we need to support, and factor that set in. We should continue to increment the baseline compilers over time, but a global "two year old compilers only" policy doesn't make sense to me. I also think that windows will be the lowest bar anyway.

Now for the carrot: if we go with this plan, then immediately after branching for 3.4, we would be able to use the vast majority of C++11 features, targeting the following as the oldest toolchains supported through the 3.5 release timeframe:

GCC 4.7
Clang 3.1
VS 2012

This seems overly aggressive to me. Why not start by baselining at VC++ 2010, and bump up everything else to match? That would give us some basic C++'11 features for 3.4, then we could bump it up to VS 2012 in 3.5 if that goes well.

-Chris

It would be nice to keep the list to things that don't require too much
support from the STL implementation. Locale support and std::thread in
the various forms are a real show stopper for bootstrapping otherwise.

Joerg

Greetings,

This has been discussed many times, and there are a lot of pro's and con's on each side, but increasingly I think the project needs to draw a line in the sand and put in place long-term policies around support for building LLVM & Clang with older C++ toolchains. It also would have the benefit of removing divergence between LLVM sub-projects already using C++11 features. As they have grown in popularity and as code moves back and forth across project boundaries, I think this is a growing strain.

Awesome, thank you for spearheading this discussion. We really need to move to some C++'11 support, it is just a question of how and when. We also had a discussion from a few months ago about this, can you summarize where we left off? I thought there was a tentative plan for LLVM 3.4 (or maybe it was after 3.4 branched).

I have a quick question on this. Apology if this has already been answered in the past. Am I correct in assuming this also force changes to other projects which link in LLVM?

Evan

Chris Lattner <clattner@apple.com> writes:

One short term caveat: Windows is special.

s/Windows/Visual Studio.

MinGW has the latest and greatest gcc.

I don't see how it is special.

It is special, sadly, and I'm not talking about C++11 support only, but
about the policies MS follows which too often makes very inconvenient
(or even impossible) to upgrade to newer VS versions. The latest example
that comes to mind was the release of VS2012: they removed Windows XP
support, as if upgrading the OS is a non-issue if you ask for it to your
users on a polite tone. An uproar followed and they backpedaled on a
service pack some months later, but that not always happens.

We should just look at all of the
compilers we need to support, and factor that set in. We should
continue to increment the baseline compilers over time, but a global
"two year old compilers only" policy doesn't make sense to me. I also
think that windows will be the lowest bar anyway.

VS 2013 introduces lots of C++11 features, but so far I've seen too many
bugs, which brings in another concern: nowadays C++03 support is fairly
mature on all recent C++ compilers but I still remember when wasting
hours trying to keep all compilers happy were a daily chore.

Now for the carrot: if we go with this plan, then immediately after
branching for 3.4, we would be able to use the vast majority of
C++11 features, targeting the following as the oldest toolchains
supported through the 3.5 release timeframe:

GCC 4.7
Clang 3.1
VS 2012

This seems overly aggressive to me. Why not start by baselining at
VC++ 2010, and bump up everything else to match?

IIRC the only significant difference among VS 2012 and VS 2010 is range
for loops.

[snip]

Hi folks,

I’m up for it. I agree there are issues to work around for specific distros, but this is dragging for far too long. The main reasons against it until last year was: “this is not the standard yet”, and “compiler support is patchy”. I think we’re past that, now.

Hi,
Ubuntu 13.10 is released and shipping g++-4.8.1. Ubuntu 14.04 is LTS and will support
g++-4.8.1. g++-4.8.1 claims to be fully compliant with C++11.

My understanding is g++4.7 is broken. I would use 4.7.2 or later.

enjoy,
Karen

libstdc++ 4.4 is extremely broken with respect to C++11 code, to the point that using clang 3.3, libstdc++, -std=c++11, and #include <vector> is a nightmare to even work around. I discovered this recently when having to make such code compile on an RHEL system.

Chris Lattner <clattner@apple.com> writes:

One short term caveat: Windows is special.

s/Windows/Visual Studio.

MinGW has the latest and greatest gcc.

Sometimes, MSVC works better than gcc. Mozilla is attempting to switch to char16_t and we have discovered that this requires making life either moderately painful or extremely painful for MingW. (Admittedly, this is an extreme case, but it goes to show you that you need to also take into account the interaction between platform headers and compiler support)

IIRC the only significant difference among VS 2012 and VS 2010 is range
for loops.

enum class, override/final keywords. I have a nice chart of when things were added to major compilers at <https://developer.mozilla.org/en-US/docs/Using_CXX_in_Mozilla_code&gt;\.

I discovered this recently when having to
make such code compile on an RHEL system.

RHEL seems to be responsible for an awful lot of pain. Perhaps it
should be considered merely a client rather than any significant
driver of our actions: making appropriate versions of real software
that's actually moving available to those who want to use 10 year old
releases.

Tim.

Yes.

-Chris