LLVM 3.4.1 - Testing Phase

Hi,

I have just tagged the first release candidate for the
3.4.1 release, so testers may begin testing. Please refer to
http://llvm.org/docs/ReleaseProcess.html for information on how to
validate a release. If you have any questions or need
something clarified, just email the list.

For the 3.4.1 release we want to compare test results against 3.4-final.

I have added support to the test-release.sh script for dot releases and
have done basic testing. However, if you run into issues please send
an email to the list and we can try to get it worked out.

When you report your test results to me please remind me of what platform
you are testing on.

Thanks again to all the testers for your help.

-Tom

OpenSUSE 13.1 i586 looks good
OpenSUSE 13.1 x86_64 fails to compile driver.c from MultiSource/Applications/sgefa saying:

/tmp/driver-becf24.s: Assembler messages:

/tmp/driver-becf24.s:857: Error: operand type mismatch for `pshufd’

Binaries uploaded to ftp.

Fedora 20 binaries uploaded, same results as with openSUSE, sgefa fails on 64bit version.

The Debian & Ubuntu packages built from the 3.4.1 branches are working
fine with tests:
http://llvm.org/apt/

(Sorry about the delay, I had a dead hard drive on one of my build systems).

So, good work Tom!

Cheers,
Sylvestre

Hi Tom,

Testing on OS X 10.9 was successful, e.g. no errors. Uploaded the archive to https://www.dropbox.com/s/lbz4gpzgjtbirkg/clang%2Bllvm-3.4.1-rc1-x86_64-apple-darwin10.9.tar.xz (sorry cannot access the FTP currently, the SSH key is on another machine), MD5: db2671feebe6312c055ce68e436627fa

Cheers,
Sebastian

Hi All,

As kindly pointed by Sebastian, please forget my point about the release script. I read the mail from Tom 2 days ago, and forgot that the script was available in the tag / branch when I acted today :frowning:

I uploaded clang+llvm-3.4.1-rc1-x86_64-unknown-ubuntu12.04.tar.xz to ftp.llvm.org.

All tests are OK but:

  • voronoi (usual failure: not a regression)
  • sgefa (new: regression): “assembler message: Error: operand type mismatch for ‘pshufd’”

Cheers,

I do not see this failure in my regular test suite builds after my for to voronoi in 196186. Did you use the latest test-suite version?

Tobias

I used test-suite as tagged for 3.4 final — in order to detect regressions — so r196186 should be there. This was all done with clean checkouts & directories, but I will double-check tomorrow my setup.

(Re-posting from my subscribed address. Sorry for the duplicate.)

Hi,

I have just tagged the first release candidate for the
3.4.1 release, so testers may begin testing. Please refer to
http://llvm.org/docs/ReleaseProcess.html for information on how to
validate a release. If you have any questions or need
something clarified, just email the list.

I tried to do a 3.4.1-rc1 Windows build using the same script I used
for the main 3.4 release, but I noticed that compiler-rt, lld and
clang-tools-extra didn't have 3.4.1 tags, so I used 3.4-final for
those.

The binary is available here
http://www.hanshq.net/LLVM-3.4.1-rc1-win32.exe (sha1:
1773f90a81d3fc569dd27a8d3ffa3407abf88c79). I've pinged Anton for sftp
access, so hopefully I can use that for the next one.

Cheers,
Hans

I have just tagged the first release candidate for the
3.4.1 release, so testers may begin testing. Please refer to
http://llvm.org/docs/ReleaseProcess.html for information on how to
validate a release. If you have any questions or need
something clarified, just email the list.

Hi Tom,

Sorry for the delay, I'm on holidays. Just to clarify, that document
is slightly outdated, I need to update it. But the bottom line is that
the script downloads the correct sources and it already creates the
release package.

I have added support to the test-release.sh script for dot releases and
have done basic testing. However, if you run into issues please send
an email to the list and we can try to get it worked out.

Is this on trunk or on the release_34 branch?

When you report your test results to me please remind me of what platform
you are testing on.

I'll run on ARMv7A Cortex-A15 Hard-float Linux
(armv7a-linux-gnueabihf). It may take a while...

cheers,
--renato

My setup is fine. As said, I used test-suite 3.4-final for testing for regressions.

It appears that:

  • r196186: voronoi: Make printing of negative NaNs consistent… (your patch)
  • r195646: Work-around for ClamAV uncertainty in test-suite (by Renato)
    have not been backported to the RELEASE_34 branch

> I have just tagged the first release candidate for the
> 3.4.1 release, so testers may begin testing. Please refer to
> http://llvm.org/docs/ReleaseProcess.html for information on how to
> validate a release. If you have any questions or need
> something clarified, just email the list.

Hi Tom,

Sorry for the delay, I'm on holidays. Just to clarify, that document
is slightly outdated, I need to update it. But the bottom line is that
the script downloads the correct sources and it already creates the
release package.

> I have added support to the test-release.sh script for dot releases and
> have done basic testing. However, if you run into issues please send
> an email to the list and we can try to get it worked out.

Is this on trunk or on the release_34 branch?

dot release aware test-release.sh script is in the release_34 branch.

I think we should. This has zero impact on anything else. But again,
it doesn't matter, we can easily ignore those results.

cheers,
--renato

> I have just tagged the first release candidate for the
> 3.4.1 release, so testers may begin testing. Please refer to
> http://llvm.org/docs/ReleaseProcess.html for information on how to
> validate a release. If you have any questions or need
> something clarified, just email the list.

Hi Tom,

Sorry for the delay, I'm on holidays. Just to clarify, that document
is slightly outdated, I need to update it. But the bottom line is that
the script downloads the correct sources and it already creates the
release package.

> I have added support to the test-release.sh script for dot releases and
> have done basic testing. However, if you run into issues please send
> an email to the list and we can try to get it worked out.

Is this on trunk or on the release_34 branch?

This is in the release_34 branch.

-Tom

Hi Tom,

Binary for armv7-unknown-linux-gnueabihf can be found at:

https://www.dropbox.com/s/uhetjgzlmlxz5mq/clang%2Bllvm-3.4.1-rc1-armv7-unknown-linux-gnueabihf.tar.xz

Compared to 3.4, llvm.check for phase 3 has the following differences:
- new failure: ExecutionEngine/MCJIT/remote/simpletest-remote.ll
- new unexpected pass: ExecutionEngine/MCJIT/remote/cross-module-sm-pic-a.ll

Cheers,
Erik.

I've failed a little bit on timing, and I will defer to Sylvestre for the builds, since they are nicely packaged.

Here are my results on Ubuntu Saucy, compared to my installed 3.4 (clang version 3.4 (tags/RELEASE_34/final)):

= Test Summary =
New Failures 2
Performance Regressions 198
Performance Improvements 79
Existing Failures 120
Unchanged Tests 593
Total Tests 992

All the failures are compile or execution time.

The new failures are compile/execution time for
MultiSource/Applications/sgefa/sgefa

If I compare it to my previous run of the 3.4 built using the test-release (clang version 3.4 (tags/RELEASE_34/final)):

= Tests Summary =
New Failures 122
New Passes 1
Performance Regressions 52
Performance Improvements 379
Added Tests 2
Unchanged Tests 436
Total Tests 992

The new pass is execution time for:
MultiSource/Benchmarks/Olden/voronoi/voronoi

I'm not sure why there are so many failures on compile and execution time.

Ben

Test results are starting to come in, it looks like we have a few
regressions:

x86_64:
MultiSource/Applications/sgefa

ARMV7:
- new failure: ExecutionEngine/MCJIT/remote/simpletest-remote.ll
- new unexpected pass:
  ExecutionEngine/MCJIT/remote/cross-module-sm-pic-a.ll

Has anyone attempted to bisect to figure out which commit broke these
tests?

-Tom

Don't hold the release for these. They're known to be unstable and I
though we had fixed them, but clearly there's still room to improve.

I just ran the release and all failures are in the execution engine,
which is ok for now to be broken. No point in bisecting, since it
fails randomly.

I'll run the test-suite now...

cheers,
--renato

Test-suite is green, only failing EH tests, but 3.4 didn't support EH
by default.

We're good to go on the ARM+Linux side. Do you have the binaries or
should I upload it on the FTP site?

cheers,
--renato

> I'll run the test-suite now...

Test-suite is green, only failing EH tests, but 3.4 didn't support EH
by default.

We're good to go on the ARM+Linux side. Do you have the binaries or
should I upload it on the FTP site?

Go ahead and upload your binaries.

-Tom