make check failures

Clang Frontend
1

I've just tried "make check" on a fresh checkout of llvm and clang. I get:

Failing Tests (26):
    LLVM :: FrontendC++/2010-03-22-empty-baseclass.cpp
    LLVM :: FrontendC++/2010-04-30-OptimizedMethod-Dbg.cpp
    LLVM :: FrontendC++/2010-05-11-alwaysinlineinstantiation.cpp
    LLVM :: FrontendC++/2010-06-22-BitfieldInit.cpp
    LLVM :: FrontendC++/thunk-linkonce-odr.cpp
    LLVM :: FrontendC/2007-04-11-InlineStorageClassC89.c
    LLVM :: FrontendC/2007-04-11-InlineStorageClassC99.c
    LLVM :: FrontendC/2007-05-16-EmptyStruct.c
    LLVM :: FrontendC/2007-10-01-BuildArrayRef.c
    LLVM :: FrontendC/2010-05-14-Optimized-VarType.c
    LLVM :: FrontendC/2010-05-18-asmsched.c
    LLVM :: FrontendC/2010-05-18-palignr.c
    LLVM :: FrontendC/2010-05-26-AsmSideEffect.c
    LLVM :: FrontendC/2010-06-11-SaveExpr.c
    LLVM :: FrontendC/2010-06-17-asmcrash.c
    LLVM :: FrontendC/2010-06-28-DbgLocalVar.c
    LLVM :: FrontendC/2010-07-08-DeclDebugLineNo.c
    LLVM :: FrontendC/2010-07-27-MinNoFoldConst.c
    LLVM :: FrontendC/2010-11-16-asmblock.c
    LLVM :: FrontendC/arrayderef.c
    LLVM :: FrontendC/asm-reg-var-local.c
    LLVM :: FrontendC/crash-invalid-array.c
    LLVM :: FrontendC/inline-asm-function.c
    LLVM :: FrontendC/pr2394.c
    LLVM :: FrontendC/vla-1.c
    LLVM :: FrontendC/vla-2.c

  Expected Passes : 5554
  Expected Failures : 53
  Unsupported Tests : 69
  Unexpected Failures: 26

Is this normal? I configured with just --enable-optimized.
BUILD_TRIPLE=x86_64-unknown-linux-gnu.

Thanks,
Jay.

2

The LLVM frontend tests are a disaster.

I recommend configuring with --without-llvmgcc and --without-llvmgxx.

- Daniel

3

I've just tried "make check" on a fresh checkout of llvm and clang. I get:

These tests are actually testing llvm-gcc, not llvm. You can almost certainly get them to pass by building llvm-gcc and keeping it up to date. If you don't want to do that, you can configure using --without-llvm-gcc as Daniel suggested.

People often react to discovering this behavior by suggesting gently that llvm-gcc tests belong in llvm-gcc, not in llvm. So far no one has been inspired to do the work of moving them, however. The test harnesses are sufficiently different that this is not trivial.

4

Thanks. That wasn't at all obvious to me!

Jay.

5

When I configure LLVM --with-llvmgcc --with-llvmgxx, make check-all
reports loads of stuff ending with:

    [...]
    LLVM :: Transforms/PartialSpecialize/heuristics.ll
    LLVM :: Transforms/SCCP/ipsccp-addr-taken.ll
    LLVM :: Transforms/ScalarRepl/memcpy-from-global.ll
    LLVM :: Transforms/ScalarRepl/vector_promote.ll
    LLVM :: Transforms/SimplifyCFG/basictest.ll
    LLVM :: Transforms/SimplifyCFG/indirectbr.ll
    LLVM :: Transforms/SimplifyCFG/speculate-with-offset.ll
    LLVM :: Transforms/SimplifyLibCalls/Puts.ll
    LLVM :: Transforms/SimplifyLibCalls/StrPBrk.ll
    LLVM :: Transforms/SimplifyLibCalls/StrRChr.ll
    LLVM :: Transforms/SimplifyLibCalls/StrSpn.ll
    LLVM :: Verifier/2010-08-07-PointerIntrinsic.ll

  Expected Passes : 7684
  Expected Failures : 69
  Unsupported Tests : 69
  Unexpected Passes : 3
  Unexpected Failures: 571
make[1]: *** [check-local-all] Error 1

Here's one of the failures, picked at random:

6

I've just tried "make check" on a fresh checkout of llvm and clang. I get:

These tests are actually testing llvm-gcc, not llvm. You can almost certainly get them to pass by building llvm-gcc and keeping it up to date.

When I configure LLVM --with-llvmgcc --with-llvmgxx, make check-all
reports loads of stuff ending with:

   [...]
   LLVM :: Transforms/ScalarRepl/vector_promote.ll
   [...]

Here's one of the failures, picked at random:

********************
FAIL: LLVM :: Transforms/ScalarRepl/vector_promote.ll (7914 of 8396)
******************** TEST 'LLVM ::
Transforms/ScalarRepl/vector_promote.ll' FAILED ********************
Script:
--
opt < /home/jay/svn/llvm-project/llvm/trunk/test/Transforms/ScalarRepl/vector_promote.ll
-scalarrepl -S | FileCheck
/home/jay/svn/llvm-project/llvm/trunk/test/Transforms/ScalarRepl/vector_promote.ll
--
Exit Code: 1
Command Output (stderr):
--
/home/jay/svn/llvm-project/llvm/trunk/test/Transforms/ScalarRepl/vector_promote.ll:99:15:
error: expected string not found in input
; CHECK-NEXT: bitcast <2 x float> %X to i64
             ^
<stdin>:48:18: note: scanning from here
define i64 @test6(<2 x float> %X) {
                ^
<stdin>:49:9: note: possible intended match here
%tmp = bitcast <2 x float> %X to <1 x i64> ; <<1 x i64>> [#uses=1]
       ^
--
Is this the "disaster" of which Daniel spake?

I don't think so, llvm-gcc is not involved at all in running that test. This looks more like you're picking up an old version of "opt" somehow. The output for this test changed about a month ago and the result you're getting is consistent with what it looked like before the change.

7

I eventually worked out what was going on.

I configured llvm --with-llvmgcc=.../install/llvm-gcc. In that same
install directory was an old binary of "opt" which I had carelessly
left there from another install ages ago.

"make check" seems to use any "opt" which it happens to find in the
same directory as llvm-gcc, in preference to the "opt" which you have
just built, which seems a bit unfriendly.

Thanks,
Jay.