> Please take a look and send feedback toi the list. I'd like to get
> process moving early in the 2.4 cycle.
Here are my opinions:
I like the idea of regular validation tagging. However, I think that
it should be as automated as possible. I'm worried that validation
testing will be pushed off of people's plates indefinitely – even for
people who care deeply about a particular platform.
Yes, automation is key. I think it is very possible to do with this proposal.
Here are the minimal set of tests that should go into a validation
test. (All of these should be done in "Release" mode.)
* Regression Tests - This catches obvious errors, but not major ones.
For clarity, you mean "make check" on the LLVM tools, right?
* Full Bootstrap of LLVM-GCC - LLVM-GCC is a complex program. It's our
second indication if something has gone horribly awry.
* Nightly Testsuite - A very good suite of tests; much more extensive
than a simple bootstrap.
How does this differ from "make check" or llvm-test?
* LLVM-GCC Testsuite - Many thousands of great tests to test the many
facets of the compiler. WAY too few people run these.
By this do you mean llvm-test or the testsuite that ships with gcc? To
my knowledge, LLVM has never passed the gcc testsuite ("make check"
As far as I know, Dale's the only one who's been slogging through the
LLVM-GCC testsuite, finding and fixing errors. I think that there are
still many failures that should be addressed.
Depending on how you're defining LLVM-GCC I may also be running those tests
All four of the above should be run on at least a nightly basis (more
frequently for some, like the regression tests). Each of these are
automated, making that easy. If there are no regressions from the
above four, we could tag that revision as being potentially "valid".
Right. I would add one thing. We want to run these suites with Debug,
Release, Release+Asserts and Debug+ExpensiveChecks builds. No one
but me seems to run Debug+ExpensiveChecks tests because I see things
break regularly. It's a valuable tool to find subtle C++ errors.