Hi All!
Replying to John here on twitter:
https://twitter.com/echristo/status/1192506726920011776
where he was noticing the helpfulness of -ferror-limit=1 as making a
huge usability difference. I acknowledge that being able to do good
error recovery and keep going is useful in some cases, but for day to
day interactive this might be a better usability default? Especially
with the compiler fast enough for the edit/compile cycle?
Thoughts? Shall we do this?
-eric
From: cfe-dev <cfe-dev-bounces@lists.llvm.org> On Behalf Of Eric
Christopher via cfe-dev
Sent: Thursday, November 7, 2019 1:23 PM
To: Clang Dev <cfe-dev@lists.llvm.org>
Cc: Richard Smith <richardsmith@google.com>
Subject: [cfe-dev] -ferror-limit=1 by default?Hi All!
Replying to John here on twitter:
https://twitter.com/echristo/status/1192506726920011776
where he was noticing the helpfulness of -ferror-limit=1 as making a
huge usability difference. I acknowledge that being able to do good
error recovery and keep going is useful in some cases, but for day to
day interactive this might be a better usability default? Especially
with the compiler fast enough for the edit/compile cycle?Thoughts? Shall we do this?
This feels like a case where interactive use and build-system use
might want different defaults, because the overhead of a starting up
a big build is typically a lot higher. Would this be an imposition
on all the build systems out there, that for usability they would have
to set a higher error-limit?
--paulr
I’d vote against it, personally - I think I often look at more than one error at a time - especially with an API change/rename/reordering of function arguments/etc - I can go through at least a handful of the call site updates in one pass.
(we /might/ be able to do some experiments at Google scale with this sort of thing (if we found a good way around the build caching issues (eg: could run a sample based on file names so the cache didn’t thrash due to actually random changes in output)) & see if developers take less or more time on average to converge on a successful build)
Taking a compiler-development look at the “30 pages of crap” to see where it goes so far off the rails as to be unhelpful might be useful. I know Richard’s (& maybe Richard Trieu and other folks too) done some things/is looking into ways to be more selective about overload candidates and template instantiation stacks, etc.
Responding to you and Dave at once because you brought up similar issues:
I can definitely see that IDEs/larger build systems might want to set
a different value versus the command line compilation. I'm definitely
looking at changing the default and not removing the option 
The Google scale experiment or the refactoring change is a good
counter argument in a lot of ways - I've done that a lot in my own use
of the compiler and knowing the next 10 or so cases is pretty useful.
That said, I'm not certain it's more or less useful than "simple"
clean errors that are easy to spot in the terminal rather than having
to scroll to more of them.
I've definitely gotten a bunch of comments that go with "what about
tooling uses" which I'll count refactoring under and a number of
people wanting just the first error in my earlier thread.
Seems to be a decent set of things. I'm uncertain how much experiment
we want to run here - but I could see it being a useful change to see
what people think.
-eric
Note that there also is a -Wfatal-errors option.
Michael
There is! I think one of the things I like is that -ferror-limit also
says "if you'd like more warnings then you can use a different option"
rather than something like -Wfatal-errors.
Also thinking about usability defaults for the greater good rather
than my use case. I set my own command line options like I like
-eric
Without having an opinion on what should be the default, I find
-Wfatal-errors more idiomatic. Errors after the first are not robust
and I have no use case for "I need exactly 2 errors". Rather, if the
following errors might be useful, I'd remove the -Wfatal-errors and
less/scroll though the entire list.
Michael
Anecdotally, I've set -ferror-limit=3 over the last six months for a few projects that I work on. A few observations:
- The compile time for non-trivial C++ compilation units is sufficiently long that an extra compile cycle before getting to the second error is a noticeable delay.
- With a limit of 3 (picked in a completely non-scientific way), I have a roughly 50:50 chance of being able to fix all three errors at the same time and a higher chance of being able to fix 2.
- Occasionally, I will have 20 instances of the same root-cause. If an API changes in a header, it's useful to find all callees. I think we could do something useful with heuristics to catch things like this (for example, if the same invalid identifier appears in multiple places in what appears to be the same kind of scope, report them together). With -ferror-limit=3 currently, I have multiple compile cycles to find them all (typically, I just do one build with no error limit in these cases today).
- If I miss a closing brace, 0 of the errors that clang reports are useful, at any error limit. A single check that counts the number of open and close braces and, if they are different numbers with a fast response would be very helpful. At the moment, it can be 10-20 seconds for a heavily templated file before I see an error that is caused by something that the lexer could have seen (all of the errors are then nonsense, because they're caused by trying to parse a method definition inside another method or similar).
TL;DR: An error limit of 3 gives me too many errors some times and not enough at other times, I think a limit of 1 would give not enough a lot more often.
David
We work pretty hard to make errors after the first as robust as possible; if the user experience is that our recovery from errors in some particular case is bad, we should try to fix that before resorting to a default of -Wfatal-errors / -ferror-limit=1.
We also have some unusual cases where the first error reported is actually a consequence of a later error (or warning) that we’ve not yet diagnosed. This can happen due to our taking the unintended path through a parse ambiguity, or (occasionally) due to our attempt to work out how to diagnose an error (particularly in typo correction) encountering further errors. Stopping at the first error can in rare cases completely hide the error message that explains what’s wrong.
I think it’s better to view this as a class of individually-fixable problems rather than a systemic problem, identify the particular cases where our error recovery is bad, and improve those cases.
For me it was less about error recovery and that they're useful, but
rather primarily that they're "too many" and that focusing on
individual ones was more helpful - also things like maybe =3 rather
than =1 or something.
At any rate, it appears the result is mixed enough that it's worth
just leaving it as is and maybe documenting behavior a bit more
clearly as a helpful option.