How long does it take to run the test-release.sh script?

We’re looking at doing some release builds on github’s new larger runners, and I’m trying to estimate how much it would cost. How long (and how many cores) does it usually take to run the test-release.sh script?

On my Mac mini it takes around 5 hours.

But I think we should discuss what we want to accomplish with the binary packages / release testing as well. Currently the test-release script runs the tests for each phase. I think that’s probably not warranted. We also build the complete project for each phase. And that also seems a bit wasteful.

But it depends on what we want to get out of it, do we find more bugs from running the tests for each phase?

A suggestion to reduce the build time would be:

Phase one: build clang, lld, binutils and maybe runtimes.
Phase two: build all with phase one and test
Phase three (if we even want to keep it - never seen a problem in three that didn’t happen in two): minimal build of clang with phase 2 and compare.

I think this would slash build times while give great coverage. Thoughts?

How many cores?

Phase One looks good. If we are only doing a minimal clang build in Phase 3, does that mean we would be packaging the Phase 2 binaries?

6C/12T I think. It’s older intel one.

Yes exactly. The point of phase three as I understand it is that we want to see that we don’t have a regression in the output from a compiler from phase 2. But as I said - I don’t know if this phase catches a lot of errors that wouldn’t we catch otherwise.