[RFC] migrating past C++11

Project Infrastructure LLVM Dev List Archives
1

Hello fans of the auto keyword!

We now have a policy on how LLVM toolchains get updated! Let’s put that policy to good use, and talk about how we’ll move all monorepo projects past C++11.

Previous Discussions

Migrate to what?

I’m only proposing that we migrate to C++14 for now. If you want to propose C++17, please do the work required by the policy. In particular, document which toolchains this would require, and what features you’d unlock. As per policy, I want to start soft-errors when building LLVM 8, so that LLVM 9 can use more than C++11.

Timeline

At the LLVM dev meeting BoF, the room already agreed to move past C++11. Late March 2019 was proposed as a time when we’d start migrating, pending large contributors’ readiness. I’m sticking to that timeline, we’ll see if everyone is ready at the end of March. I nonetheless want to get the soft-errors into the LLVM 8 branch so that we give a sufficient heads-up to developers who only compile releases.

Upsides

One clear upside of dropping older toolchains: they don’t even support C++11 very well. We have a handful of workarounds left in ADT (particularly around type traits) and I’d like to get rid of them.

The compiler versions I propose allow us to use all of C++14, which includes:

See CppReference for details.

Of these, I think polymorphic lambdas are the big feature. Of course, just like Almost Always Auto, we should use such things only where it makes sense.

Toolchains

We’re currently mandating:

  • Clang 3.1 (released 2012/05)
  • Apple Clang 3.1 (released 2012/05)
  • GCC 4.8 (released 2013/03)
  • Visual Studio 2015 (Update 3) (released 2016/06)

I propose instead:

  • Clang 3.5 (released 2014/07) to get -std=c++14 instead of -std=c++1y
  • Apple Clang 6.0 (released 2014/07) to match clang 3.5
  • GCC 5.1 (released 2015/04) because C++14 mostly came to be in GCC 5
  • Visual Studio 2017 (released 2017/03) so that we get extended constexpr and NSDMI

Version information from:

My previous attempts pointed out that WebKit / Chromium / Firefox are all past C++11 (WebKit is moving to C++17 (from C++14), Chromium started moving to C++14, Firefox uses some C++14). This means that platforms which distribute a modern browser can already bootstrap a browser. That’s a nice datapoint, but isn’t sufficient for platforms which compile / use LLVM (especially as a library).

Here’s a table from the LLVM dev meeting BoF detailing version info for distros that seemed relevant:


Release
|
Distro
|
Compiler
|
C++14 lang
|

  • | - | - | - |

    2003-10
    |
    RHEL 3
    |
    GCC 3.2
    |
    :x:
    |

    2005-02
    |
    RHEL 4
    |
    GCC 3.4
    |
    :x:
    |

    2007-03
    |
    RHEL 5
    |
    GCC 4.1
    |
    :x:
    |

    2010-11
    |
    RHEL 6
    |
    GCC 4.4
    |
    :x:
    |

    2013-05
    |
    Debian 7 wheezy
    |
    GCC 4.7.2
    |
    :x:
    |

    2013-12
    |
    RHEL 7
    |
    GCC 4.8
    |
    :x:
    |

    2015-04
    |
    Debian 8 jessie
    |
    GCC 4.9.2
    |
    :x:
    |

    2015-05
    |
    OpenBSD 5.7
    |
    LLVM 3.5
    |
    :white_check_mark:
    |

    2015-10
    |
    OpenBSD 5.8
    |
    LLVM 3.5
    |
    :white_check_mark:
    |

    2016-03
    |
    OpenBSD 5.9
    |
    LLVM 3.5
    |
    :white_check_mark:
    |

    2016-04
    |
    Ubuntu 14.04
    |
    GCC 4.8.2
    |
    :x:
    |

    2016-04
    |
    Ubuntu 16.04
    |
    GCC 5.3.1
    |
    :white_check_mark:
    |

    2016-09
    |
    OpenBSD 6.0
    |
    LLVM 3.8
    |
    :white_check_mark:
    |

    2017-04
    |
    OpenBSD 6.1
    |
    LLVM 4.0.0
    |
    :white_check_mark:
    |

    2017-06
    |
    Debian 9 stretch
    |
    GCC 6.3.0
    |
    :white_check_mark:
    |

    2017-10
    |
    Ubuntu 17.10
    |
    GCC 7.2.0
    |
    :white_check_mark:
    |

    2017-10
    |
    OpenBSD 6.2
    |
    LLVM 5.0.0
    |
    :white_check_mark:
    |

    2018-04
    |
    Ubuntu 18.04
    |
    GCC 7.3.0
    |
    :white_check_mark:
    |

    2018-04
    |
    OpenBSD 6.3
    |
    LLVM 5.0.1
    |
    :white_check_mark:
    |

    2018-10
    |
    Ubuntu 18.10
    |
    GCC 8.1.0
    |
    :white_check_mark:
    |

    2018-??
    |
    Debian 10 buster
    |
    GCC 8.1.0
    |
    :white_check_mark:
    |

The data comes from the following sources:

I haven’t documented FreeBSD / NetBSD / Fedora / MacOS / MSVC, and nobody complained at the BoF. I’d like to understand if we should care about documenting these: ideally the toolchain update policy would list which platforms need to be considered and how far back in time is relevant.

Thanks,

JF

2

This is a most excellent proposal. I know Firefox’s C++ page says that extended constexpr support in GCC 5.0, but my recollection is that the support is much improved in GCC 6.1+. GCC 6.x’s constexpr support was certainly the driving motivation for Firefox’s requiring GCC 6.1. You may want to consider bumping your GCC support level a bit; according to your table, it leaves Ubuntu 16.04 out in a cold a little bit, but maybe that’s OK?

-Nathan

3

For RHEL's sake, note that one can get newer GCC here:

4

For FreeBSD, the relevant issue version included in latest supported
releases on each branch is (since we need to be able to build the HEAD
OS version from at least the previous release.) That's currently 6.0
for both FreeBSD 11.2 and 12.0.

In the past, we've not done major toolchain upgrades on release
branches, but that changed in FreeBSD 11 and I personally expect use
to keep doing them to at least some extent. The pain of maintaining
packages on top of clang 3.4 in FreeBSD 10.4 was just too much near its
end of life.

We also support many LLVM version in our ports/package collection.
Currently 3.5, 3.8, 4.0, 5.0, 6.0, 7.0, and an erratically updated
snapshot of the head of the tree. Since we currently bundle a specific
version into the release branch, these versions don't really enter into
the picture for minimum version and they don't provide a C++ standard
library and depend on the one in the base OS.

Long story short, release branches currently have pretty up to date
clang/llvm/libc++ versions and we're certainly fine with a move to
C++14. (My guess is that C++17 would be fine as well.)

-- Brooks

5

I have no objection to the end of March timeline for C++14. This will still be a bit of a pain for us, but at the moment, I think we’re on target for the previously communicated dates. I’ll note that having a lot of warning for this (Nov to March) was a key factor in my no objection response. :slight_smile:

Philip

6

Having struggling *a lot* to implement decent is_trivially_copyable<...> support
in the LLVM codebase for gcc 4.9, I can only second this version bump.

7

Do we know that the proposed newer compilers support C++14 very well? If we encounter issues with them, how are we going to deal with that?

-Krzysztof

8

I’d expect that either we’ll either workaround the issues (e.g. not start using the broken feature), or else propose to require even newer versions. And as now, discuss the expected tradeoff between new features and requiring new compiler versions.

9

Please include MSVC in the table. While the picture on Windows is way less complicated than for *nix, it’s still a platform and toolchain that matter to a number of us in the community.

Separately, there was talk of needing to have bots that specifically use the oldest supported toolchains, otherwise we can’t genuinely promise that they are really supported (don’t want feature dependencies creeping in by accident).

–paulr

10

Separately, there was talk of needing to have bots that specifically use the
oldest supported toolchains, otherwise we can't genuinely promise that they are
really supported (don't want feature dependencies creeping in by accident).

I second that.

11

Please include MSVC in the table. While the picture on Windows is way less complicated than for *nix, it's still a platform and toolchain that matter to a number of us in the community.

What are you asking for precisely? Windows and MSVC versions aren’t as version-locked as some Linux distros are when it comes to compiler version. I’ve listed a precise MSVC version in the list (as we do in GettingStarted’s toolchain list <Getting Started with the LLVM System — LLVM 18.0.0git documentation).

Separately, there was talk of needing to have bots that specifically use the oldest supported toolchains, otherwise we can't genuinely promise that they are really supported (don't want feature dependencies creeping in by accident).

Agreed that this is a good thing to have.

12

Please include MSVC in the table. While the picture on Windows is way less complicated than for *nix, it’s still a platform and toolchain that matter to a number of us in the community.

What are you asking for precisely? Windows and MSVC versions aren’t as version-locked as some Linux distros are when it comes to compiler version. I’ve listed a precise MSVC version in the list (as we do in GettingStarted’s toolchain list).

Ah, sorry, the colorful table pulled my eye away from the proposed list of toolchain versions. I see it now.

–paulr

13

Gotcha! I’ll avoid color next time. I copy / pasted from my BoF slides. :slightly_smiling_face:
FWIW, I didn’t bother adding macOS / Xcode to the table either for the same reasons (as well as macOS / Xcode being very cheap, meaning that people typically don’t lag behind much).

14

I think this proposal looks really good – I’m all in favor of making the soft-error change immediately for trunk and LLVM 8 branch, and targeting end of march for the cutover to a hard error and allowing C++14 features.

However… and I must admit to feeling embarrassed to have to say this, since Google is probably going to be the very last holdout…but…I’m not sure Google will actually be able to require C++14 support by the planned date at end of March. We’re very much working on it, but I don’t want everyone to be surprised and upset if, come March 30, we’re still not quite ready to switch over.

So, let’s move forward with end-of-march as the current proposal, anyhow, but I’d just like to ask for forgiveness in advance in case we request a further delay as that date gets closer. (FTR, I’d expect any future version increase proposals to see no objections from Google; this is a one-time-only sort of a problem for us.)

15

I think this proposal looks really good – I’m all in favor of making the soft-error change immediately for trunk and LLVM 8 branch, and targeting end of march for the cutover to a hard error and allowing C++14 features.

Great, I’ll send a PR soon and we can see concrete details there.

However… and I must admit to feeling embarrassed to have to say this, since Google is probably going to be the very last holdout…but…I’m not sure Google will actually be able to require C++14 support by the planned date at end of March. We’re very much working on it, but I don’t want everyone to be surprised and upset if, come March 30, we’re still not quite ready to switch over.

Thanks for letting us know. I think that’s totally fine: I’d much rather start the process and soft-error in LLVM 8 now, and see if everyone is ready in March. Y’all told us at the BoF that end of March wasn’t a sure thing, and from what I’ve heard you’ve been hard at work so I see no problem waiting to do the actual move.

So, let’s move forward with end-of-march as the current proposal, anyhow, but I’d just like to ask for forgiveness in advance in case we request a further delay as that date gets closer. (FTR, I’d expect any future version increase proposals to see no objections from Google; this is a one-time-only sort of a problem for us.

Sgtm

16

Note that when we discussed moderate use of `auto` recently, there was no consensus that auto-in-lambdas (ie polymorphic lambdas) should be permitted. It's kind of strange to see it hailed as a 'big feature' if consensus is not that it should be permitted.

Is there a need for more guidelines about this or any other of these newly-unlocked features?

Thanks,

Stephen.

17

IMHO the difference between “supported” and having a bot validating the configuration is whether we accept patches to fix issues on this particular platform.

Otherwise, I believe historically it has been up to the users that care about a platform to provide CI ressources for it. Do we have an alternative plan?

18

If we claim to support a Thing, then we should accept patches to fix when the Thing breaks. Whether a bot verifies that the Thing works, is not really relevant; “support” means “we say this works and will fix it when it breaks.”

A bot is a service to the community in that it can tell you more-or-less promptly when you have broken a Thing. It is not the only way to determine that you have broken a Thing. However, it does tell you that you have broken a Thing that somebody thinks is worth putting up a bot to verify that it stays not-broken.

Given the number of times lately that newcomers have had difficulty getting started (because some Thing we claim works, actually doesn’t), I think it would be valuable to the ongoing health of the project to have bots verifying the particular Thing that is the minimum supported compiler versions. This doesn’t mean I’m volunteering to provide that resource; neither does it mean I refuse to provide that resource. I am pointing out that it would be valuable generally, and as part of this longer-term goal to specify minimum compiler versions, it would be a Good Thing™ to be able to tell promptly when we’ve accidentally broken that, so that newcomers don’t beat their head against a wall and go away disgusted with an open-source project that lies to them about supported configurations.

–paulr

19

I re-read my email and I mistyped it (sorry was in a bus), what you wrote is exactly what I intended to mean. Thanks Paul.

20

If we claim to support a Thing, then we should accept patches to fix when the Thing breaks. Whether a bot verifies that the Thing works, is not really relevant; "support" means "we say this works and will fix it when it breaks."

A bot is a service to the community in that it can tell you more-or-less promptly when you have broken a Thing. It is not the only way to determine that you have broken a Thing. However, it does tell you that you have broken a Thing that somebody thinks is worth putting up a bot to verify that it stays not-broken.
Given the number of times lately that newcomers have had difficulty getting started (because some Thing we claim works, actually doesn't), I think it would be valuable to the ongoing health of the project to have bots verifying the particular Thing that is the minimum supported compiler versions. This doesn't mean I'm volunteering to provide that resource; neither does it mean I refuse to provide that resource. I am pointing out that it would be valuable generally, and as part of this longer-term goal to specify minimum compiler versions, it would be a Good Thing™ to be able to tell promptly when we've accidentally broken that, so that newcomers don't beat their head against a wall and go away disgusted with an open-source project that lies to them about supported configurations.

I agree that what you describe would be good. I don’t see why it’s relevant to this discussion: we’re talking about changing the minimum toolchain version. Whether there are bots running the minimum toolchain version today is a fact disassociated from any proposed minimum change. Indeed, the policy we just adopted (and which I’m following) doesn’t mention bots.

For sure if someone has a bot running older toolchains we want them to upgrade to the new minimum, so this RFC and soft-error will give them proper heads up. I hope they’ll chime in if they have concerns or suggestions.

I therefore think that you want to fork this discussion point into a separate email thread, since it’ll help it move this RFC forward without distraction.