RFC: switch default C language mode from gnu99 to gnu11

Clang Frontend
1

I’d like to switch the default language mode for C compilations from gnu99 to gnu11. The only effect of this change would be to turn some extension warnings off, change the default value of STDC_VERSION, and change the default value of some __has_feature(…) checks; we allow all of C11 as an extension in C99 mode anyway.

Also of note: GCC trunk just switched from C89 to C11 by default.

2

(Sorry, hit send too soon)

Please shout now if you think this is a bad idea for some reason. If no-one objects, I’ll make this change on Friday.

3

From: "Richard Smith" <richard@metafoo.co.uk>
To: "cfe-dev@cs.uiuc.edu Developers" <cfe-dev@cs.uiuc.edu>
Sent: Wednesday, October 15, 2014 5:22:48 PM
Subject: [cfe-dev] RFC: switch default C language mode from gnu99 to gnu11

I'd like to switch the default language mode for C compilations from
gnu99 to gnu11. The only effect of this change would be to turn some
extension warnings off, change the default value of
__STDC_VERSION__, and change the default value of some
__has_feature(...) checks; we allow all of C11 as an extension in
C99 mode anyway.

+1 (given that we now have a stdatomic.h setup, I think this makes sense)

-Hal

4

I'm fine with this change.

5

Me too, this overdue.

-Chris

6

The only reason GCC took so long to switch from gnu89 was that the inline keyword changed meanings between gnu89 and gnu99. As I understand it, all gnu99 code is also valid gnu11 code, so this shouldn't be a problem.

It would be nice to have a warning that objects if you use any reserved identifiers (double underscore, underscore-capital) outside of a system header / system library so that people can ensure that their c11 code will be valid for whatever the next iteration of the standard is.

David

7

I'd like to switch the default language mode for C compilations from gnu99 to gnu11. The only effect of this change would be to turn some extension warnings off, change the default value of __STDC_VERSION__, and change the default value of some __has_feature(...) checks; we allow all of C11 as an extension in C99 mode anyway.

+1

It would be nice to have a warning that objects if you use any reserved identifiers (double underscore, underscore-capital) outside of a system header / system library so that people can ensure that their c11 code will be valid for whatever the next iteration of the standard is.

Coincidentally, I just had a conversation about this very subject
today at LPC. Indeed, such a warning would be interesting to have.

cheers,
--renato

8

+1

~Aaron

9

I have an additional suggestion: we treat tests that do not explicitly specify a language standard as testing all language modes. By default, we continue to run them only in the default language mode of the clang driver, but we:

– Add a driver and cc1 flag to set a default language mode for each language
– Add a buildbot that runs all tests with the default language mode set to each possible value

This will prevent us from losing most of our C99 test coverage when we switch our default (and would also prevent us from losing most of our C++98 test coverage if/when we switch our default C++ language mode to C++11). The added driver flag might also be useful to people who aren’t ready for our new default yet.

For the new flag, how about -std-default=c99 ?

10

If you're going to do this, I would *much* rather see the test
infrastructure changed so that all are exercised by doing 'check-clang'. I
think it would be really bad to create yet another trap for developers
where tests pass locally but fail on the bots. =/

If its too expensive to do this for *all* tests w/o a standard specified,
we should cherry pick the ones we want to run in all modes and just
replicate the RUN lines with a script.

11

I suspect that: (a) the ones we want to run in all modes is "all tests,
unless otherwise specified", and (b) it is too expensive to do so by
default (it would increase our testing time by probably around 4x). I think
the biggest risk for failure-on-bots-but-not-locally would be when
developers add new tests that accidentally rely on a particular language
standard. To that end, we could possibly run C tests in C89 and C++ tests
in C++98, independent of the driver's default mode, if we're not in "run in
all modes" mode. I'm open to suggestions.

12

I suspect that: (a) the ones we want to run in all modes is "all tests,
unless otherwise specified",

I'm not sure why. CodeGen certainly doesn't seem very interesting here?

and (b) it is too expensive to do so by default (it would increase our
testing time by probably around 4x).

I'm not sure it would.

I think the biggest risk for failure-on-bots-but-not-locally would be when
developers add new tests that accidentally rely on a particular language
standard. To that end, we could possibly run C tests in C89 and C++ tests
in C++98, independent of the driver's default mode, if we're not in "run in
all modes" mode. I'm open to suggestions.

I just don't think this is a good way to get coverage. Either getting
coverage is important, in which case we should figure out which tests to
run in multiple modes, or it isn't and we shouldn't bother with it. Having
the build bots use magic configurations to waste time exhaustively
searching for compile modes which don't work seems like a bad approach. =/

13

-- Add a buildbot that runs all tests with the default language mode set
to each possible value

If you're going to do this, I would *much* rather see the test
infrastructure changed so that all are exercised by doing 'check-clang'. I
think it would be really bad to create yet another trap for developers
where tests pass locally but fail on the bots. =/

I was going to say, this seems similar to the idea we've had to run all the
LLVM tests (& maybe Clang codegen) with all the supported targets - the
tests are meant to be portable (& fail on buildbots when they aren't) and
could actually be tested locally on the same machine if we had a mode in
which we could run that.

But I'm not sure it should be the default - it seems pretty low-value to
run all these flavors for every test run, yet nice to have the option
should there be a bot failure (or to reduce the unique coverage from having
different achitecture bots - most of it should be testable on just a single
machine) to easily reproduce it locally, or do some more thorough testing
if there's otherwise reason to believe there's a portability/language
version-y problem.

(apologies for conflating these two ideas if you believe there's a good
reason for them to be handled differently, but just where my mind went at
the mention of this stuff)

Essentially I see running all these flavors as akin to running the test
suite - it's an extra level of testing but I wouldn't make it the default
as I don't think it hits the cost/benefit tradeoff for interactive
development. Numbers would be necessary/helpful to back that up (or refute
it).

- David

14

I suspect that: (a) the ones we want to run in all modes is "all tests,
unless otherwise specified",

I'm not sure why. CodeGen certainly doesn't seem very interesting here?

Why not? If we start generating wrong code in C++11 mode (due to a subtle
change in AST representation in that mode), I want our tests for $feature
to catch that. I can't think of *any* tests that should not run in all
modes, other than those which are testing the behavior of a particular mode.

and (b) it is too expensive to do so by default (it would increase our

testing time by probably around 4x).

I'm not sure it would.

This is obviously predicated on (a).

I think the biggest risk for failure-on-bots-but-not-locally would be when

developers add new tests that accidentally rely on a particular language
standard. To that end, we could possibly run C tests in C89 and C++ tests
in C++98, independent of the driver's default mode, if we're not in "run in
all modes" mode. I'm open to suggestions.

I just don't think this is a good way to get coverage. Either getting
coverage is important, in which case we should figure out which tests to
run in multiple modes, or it isn't and we shouldn't bother with it. Having
the build bots use magic configurations to waste time exhaustively
searching for compile modes which don't work seems like a bad approach. =/

Having to explicitly write RUN: lines for all languages for all tests seems
like a bad approach too. I'm claiming that (1) getting coverage is probably
important, and (2) figuring out which tests to run will probably result in
"all tests that don't specify otherwise".

15

If all of these tests are really important to be running in all of the
modes, then all of the developers should be running them in those modes
too. I'm really concerned about increasing significantly how many different
ways a developer is expected to test their code prior to submission. It
makes the entire development process a lot more hazy, and it is already
hazy due to portability issues and the runtime testing of the test suite.

16

The question isn’t the absolute amount of time it adds, the question is whether the added time is paid for by additional bugs found. I can’t imagine that testing all tests as C11 and C99 would find many bugs. The standards are too similar, and therefore doing this by default would just be a waste of testing time.

-Chris

17

+1 for cherry-picking. Not all tests make sense to run in all
standards (C11 ones, for instance), but some would certainly benefit.
Maybe the ones that don't specify an -std flag would be good to run on
all standards? How would we do that? By changing the semantics of
%clang? Or by enforcing people to duplicate all RUN lines with all
standards?

It seems non-trivial to enforce this...

cheers,
--renato

18

I have an additional suggestion: we treat tests that do not explicitly specify a language standard as testing all language modes. By default, we continue to run them only in the default language mode of the clang driver, but we:

– Add a driver and cc1 flag to set a default language mode for each language

– Add a buildbot that runs all tests with the default language mode set to each possible value

This will prevent us from losing most of our C99 test coverage when we switch our default (and would also prevent us from losing most of our C++98 test coverage if/when we switch our default C++ language mode to C++11). The added driver flag might also be useful to people who aren’t ready for our new default yet.

For the new flag, how about -std-default=c99 ?

Seems like sufficient lit.cfg magic could vary the substitution of the %clang[_cc1] variables to supply -std=whatever. As long as Clang’s -std option parsing takes the last-one-specified without complaint, any test that explicitly specified -std would still obey that option. No clang changes needed. One bot per non-default-dialect would be sufficient.

There would be some period of pain while smoking out the tests that actually did depend on the older dialect. As a data point, when Sony privately switched to default to C++11, we added –std to something like 100-150 tests to get them to pass again, but the rest were all fine.

–paulr

19

I have an additional suggestion: we treat tests that do not explicitly
specify a language standard as testing *all* language modes. By default, we
continue to run them only in the default language mode of the clang driver,
but we:

-- Add a driver and cc1 flag to set a default language mode for each
language

-- Add a buildbot that runs all tests with the default language mode set
to each possible value

This will prevent us from losing most of our C99 test coverage when we
switch our default (and would also prevent us from losing most of our C++98
test coverage if/when we switch our default C++ language mode to C++11).
The added driver flag might also be useful to people who aren't ready for
our new default yet.

For the new flag, how about -std-default=c99 ?

Seems like sufficient lit.cfg magic could vary the substitution of the
%clang[_cc1] variables to supply -std=whatever. As long as Clang's -std
option parsing takes the last-one-specified without complaint, any test
that explicitly specified -std would still obey that option. No clang
changes needed. One bot per non-default-dialect would be sufficient.

That doesn't work: we reject -std=c99 for C++ compiles, and reject
-std=c++98 for C compiles. I believe that's the correct behaviour. What we
want to say is "use this language standard by default for C compilations"
in a way that doesn't break if the compilation is C++. That's actually a
useful feature independent of any changes to how we test Clang.

There would be some period of pain while smoking out the tests that

actually did depend on the older dialect. As a data point, when Sony
privately switched to default to C++11, we added –std to something like
100-150 tests to get them to pass again, but the rest were all fine.

That's an interesting datapoint. Did you find any bugs in this process, or
were all the changes between C++98 and C++11 modes things like extension
warnings going away?

20

I suspect that: (a) the ones we want to run in all modes is "all tests,
unless otherwise specified",

I'm not sure why. CodeGen certainly doesn't seem very interesting here?

and (b) it is too expensive to do so by default (it would increase our
testing time by probably around 4x).

I'm not sure it would.

The question isn’t the absolute amount of time it adds, the question is
whether the added time is paid for by additional bugs found. I can’t
imagine that testing all tests as C11 and C99 would find many bugs. The
standards are too similar, and therefore doing this by default would just
be a waste of testing time.

As a simple sanity test, it would make sense to hack up a prototype that
does the proposed run-in-different-language-modes-automatically with
coverage info, and then aggregate the coverage data and compare it to a
control run.

Alternatively, it would be useful to exhibit at least a bug or two that
would have been found by this extra testing (or which are uncovered by a
quick prototype).

-- Sean Silva