Straw poll: -verify and fatal errors

Hi all,

I am working on a series of patches to enhance and fix the -verify feature of clang, but one thing in particular has raised some questions which myself and Mr. Rose would appreciate some input on!

For those not familiar with the -verify feature, here is a very brief description taken from the clang internals web-page: "Instead of formatting and printing out the diagnostics, -verify just captures and remembers the diagnostics as they fly by. Then -verify compares the list of produced diagnostics to the list of expected ones. If they disagree, it prints out its own output." It is a feature commonly used in writing test cases for clang, but also has the potential to be used for non-related projects.

The question at hand here relates to how -verify should handle fatal errors generated within test cases. At the moment, -verify doesn't handle them at all: it cannot be used to check for the generation of fatal errors, nor does it provide any diagnostics back to the user.

So, which would people rather?

1. -verify should not be used to check for fatal errors but instead should halt the test case with a diagnostic direct to the user relating to the fatal error, similar to normal compiler operation

or

2. -verify should be capable of checking for the generation of fatal errors in the same way as it can for all other errors/warnings/etc.

Both routes are feasible and patches (or partial patches!) exist for both: the question is more which is the better route to take. Better can mean either from a user's / test case writer's point of view, or from a technical point of view -- or both!

Thanks for any and all feedback!

Andy

For those not following on cfe-commits, this is an inaccurate summary. We use -verify to check fatal errors in a few tests (for instance, test/SemaTemplate/instantiation-depth*), and it works, but if the test fails then -verify is unable to produce a diagnostic explaining why, because all diagnostics are suppressed after a fatal error:

$ echo ‘#include “does not exist” // expected-error {{wrong error}}’ | clang -cc1 -x c - -verify; echo $?
2 errors generated.
1
$ echo ‘#include “does not exist” // expected-error {{file not found}}’ | clang -cc1 -x c - -verify; echo $?

0
$

Yes, and I kicked myself too because I did know it but I had totally
forgotten it in all the discussions. Apologies to all for bungling
my post. So, the behaviour needs to be option 2 (i.e. treat fatal
errors as all other errors/warnings), but fix the fact that it doesn’t
produce diagnostics to the user.

Jordan, I will post the patch for this on cfe-commits this morning
when I’m properly awake (unlike last night!). I have just noted that
you’ve committed the first three parts: many thanks!!

Cheers,
Andy