POST MORTEM: llvm-test changes

Project Infrastructure LLVM Dev List Archives
21

For the heck of it I tried upgrading to gcc 3.4.2 (from 3.3.3). It
didn't make a difference. So here are the failures for llvm-test. All
diffs are against the "native" output.

===================== MultiSource/Applications/sgefa

cbe failed differently from jit/llc. First cbe:

  84c84
  < One-Norm(A) ---------- 8.879153e+02.
  ---
  > One-Norm(A) ---------- 8.879156e+02.
  100c100
  < One-Norm(A) ---------- 5.000000e+35.
  ---
  > One-Norm(A) ---------- 5.000005e+35.

And jit/llc:

  79a80
  > Zero Column 200 found

sgefa is a known XFAIL. See the nightly test results over the last
several months. Actually, you should compare your test results with the
1.3 release test results (on or about August 13th, 2004). Please only
report deltas from there.

===================== MultiSource/Applications/kimwitu++

Only llc failed. Instead of producing 13,000+ lines of parsing actions,
it produces the following two lines:

  Reading input files ...kc++: segmentation violation
  kc.llc: received signal 11, cleaning up

Not sure what's up with this. This test doesn't appear on the nightly
tester although it *should*.

===================== MultiSource/Benchmarks/Olden/power

Only cbe failed:

  6c6
  < TR=0.81, TI=0.16, P0=6397.86, Q0=1288.50
  ---
  > TR=0.81, TI=0.16, P0=6397.87, Q0=1288.50
  26c26
  < TR=0.80, TI=0.16, P0=7565.44, Q0=1525.86
  ---
  > TR=0.80, TI=0.16, P0=7565.44, Q0=1525.85
  34c34
  < TR=0.79, TI=0.16, P0=7725.99, Q0=1558.54
  ---
  > TR=0.79, TI=0.16, P0=7725.99, Q0=1558.53
  36c36
  < TR=0.79, TI=0.16, P0=8052.56, Q0=1625.04
  ---
  > TR=0.79, TI=0.16, P0=8052.56, Q0=1625.05
  74c74
  < TR=0.79, TI=0.16, P0=7894.33, Q0=1592.81
  ---
  > TR=0.79, TI=0.16, P0=7894.32, Q0=1592.81

This works on Linux (it did last night) so not sure why FreeBSD gives
these rounding errors. Can you investigate? Does the test use fpcmp?

===================== MultiSource/Benchmarks/OptimizerEval

This test continues to be non-deterministic. Sample of partial diff:

Not sure what's up with this one.

===================== MultiSource/Benchmarks/FreeBench/mason

All fail (including native) with error "Could not open datafile test.in"

When's the last time you cvs updated llvm-test, wiped out your build
directory, reconfigured, and rebuilt? There have been numerous
structural changes to llvm-test in the last week. It seems to have
settled down now and the above error is indicative of at least one of
the changes that fixed problems.

===================== MultiSource/Benchmarks/MallocBench/perl

None of cbe/jit/llc passed. gccld core dumped:

This is a known XFAIL.

===================== SingleSource/UnitTests/SetjmpLongjmp/C++/Output/SimpleC++Test

All of cbe/jit/llc failed with:

  2a3
  > Destroying C object in foo

Again, a known XFAIL.

===================== SingleSource/Regression/C++/EH/Output/ConditionalExpr

Only cbe failed with:

gcc Output/ConditionalExpr.cbe.c -std=c99 -fno-strict-aliasing -O2 -o Output/ConditionalExpr.cbe
Output/ConditionalExpr.cbe.c:1629: warning: conflicting types for built-in function 'memset'
Output/ConditionalExpr.cbe.c: In function `l141__ZNKSt7collateIwE12do_transformEPKwS2_':
Output/ConditionalExpr.cbe.c:29424: warning: implicit declaration of function `alloca'
/var/tmp//cc7mQbke.o: In function `l292__ZNKSt8time_getIwSt19istreambuf_iteratorIwSt11char_traitsIwEEE15_M_extract_nameERS3_S5_RiPPKwjRSt12_Ios_Iostate':
/var/tmp//cc7mQbke.o(.text+0x122e3): undefined reference to `alloca'
/var/tmp//cc7mQbke.o: In function `l164__ZNKSt8time_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_RSt8ios_basewPK2tmcc':
/var/tmp//cc7mQbke.o(.text+0x139aa): undefined reference to `alloca'
/var/tmp//cc7mQbke.o: In function `l311__ZNSt5__padIwSt11char_traitsIwEE6_S_padERSt8ios_basewPwPKwiib':
/var/tmp//cc7mQbke.o(.text+0x14db8): undefined reference to `alloca'
/var/tmp//cc7mQbke.o: In function `l195__ZNKSt9money_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_bRSt8ios_basewe':
/var/tmp//cc7mQbke.o(.text+0x15180): undefined reference to `alloca'
/var/tmp//cc7mQbke.o(.text+0x153a5): undefined reference to `alloca'
/var/tmp//cc7mQbke.o(.text+0x15580): more undefined references to `alloca' follow
collect2: ld returned 1 exit status
gmake[4]: [Output/ConditionalExpr.cbe] Error 1 (ignored)

You need to look into this. Why isn't alloca being linked in? Where is
it on FreeBSD? This comment goes for all the other C++/EH tests that
failed for "No alloca".

===================== SingleSource/Benchmarks/Shootout-C++/Output/echo

Fails with:

/usr/include/machine/endian.h: In function `__uint32_t __bswap32(__uint32_t)':
/usr/include/machine/endian.h:156: error: LLVM does not yet support inline assembly! Code: 'xchgb %h0, %b0
  rorl $16, %0
  xchgb %h0, %b0'
gmake[3]: [Output/echo.ll] Error 1 (ignored)

I guess I have to place a modified version of this header file in sys-include.

Yup !

===================== SingleSource/Benchmarks/Misc/Output/pi

All of cbe/jit/llc fail with:

  2c2
  < x = 0.150572 y = 0.49 low = 31314973 j = 40000001
  ---
  > x = 0.150572 y = 0.49 low = 31314970 j = 40000001

Hmm. Not sure what's up with that but Linux passes this one.

Reid.

22

>
> ===================== MultiSource/Applications/sgefa
>
sgefa is a known XFAIL. See the nightly test results over the last
several months. Actually, you should compare your test results with the
1.3 release test results (on or about August 13th, 2004). Please only
report deltas from there.

Where can I find these? I see the nightly reports, but they don't say
what's XFAIL much less /how/ they're expected to fail.

> ===================== MultiSource/Benchmarks/Olden/power
>
> Only cbe failed:
>
> 6c6
> < TR=0.81, TI=0.16, P0=6397.86, Q0=1288.50
> ---
> > TR=0.81, TI=0.16, P0=6397.87, Q0=1288.50
> 26c26
> < TR=0.80, TI=0.16, P0=7565.44, Q0=1525.86
> ---
> > TR=0.80, TI=0.16, P0=7565.44, Q0=1525.85
> 34c34
> < TR=0.79, TI=0.16, P0=7725.99, Q0=1558.54
> ---
> > TR=0.79, TI=0.16, P0=7725.99, Q0=1558.53
> 36c36
> < TR=0.79, TI=0.16, P0=8052.56, Q0=1625.04
> ---
> > TR=0.79, TI=0.16, P0=8052.56, Q0=1625.05
> 74c74
> < TR=0.79, TI=0.16, P0=7894.33, Q0=1592.81
> ---
> > TR=0.79, TI=0.16, P0=7894.32, Q0=1592.81

This works on Linux (it did last night) so not sure why FreeBSD gives
these rounding errors. Can you investigate? Does the test use fpcmp?

It does:

/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-llc Output/power.llc
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" llc power
/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-cbe Output/power.cbe
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" cbe power
/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-jit /usr/home/llvm/obj/tools/Debug/lli -force-interpreter=false Output/power.llvm.bc
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" jit power

> ===================== MultiSource/Benchmarks/FreeBench/mason
>
> All fail (including native) with error "Could not open datafile test.in"

When's the last time you cvs updated llvm-test, wiped out your build
directory, reconfigured, and rebuilt? There have been numerous
structural changes to llvm-test in the last week. It seems to have
settled down now and the above error is indicative of at least one of
the changes that fixed problems.

Last night, after I installed gcc 3.4.2. I also did a full update.

>
> ===================== SingleSource/Regression/C++/EH/Output/ConditionalExpr
>
> Only cbe failed with:
>
> gcc Output/ConditionalExpr.cbe.c -std=c99 -fno-strict-aliasing -O2 -o Output/ConditionalExpr.cbe
> Output/ConditionalExpr.cbe.c:1629: warning: conflicting types for built-in function 'memset'
> Output/ConditionalExpr.cbe.c: In function `l141__ZNKSt7collateIwE12do_transformEPKwS2_':
> Output/ConditionalExpr.cbe.c:29424: warning: implicit declaration of function `alloca'
> /var/tmp//cc7mQbke.o: In function `l292__ZNKSt8time_getIwSt19istreambuf_iteratorIwSt11char_traitsIwEEE15_M_extract_nameERS3_S5_RiPPKwjRSt12_Ios_Iostate':
> /var/tmp//cc7mQbke.o(.text+0x122e3): undefined reference to `alloca'
> /var/tmp//cc7mQbke.o: In function `l164__ZNKSt8time_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_RSt8ios_basewPK2tmcc':
> /var/tmp//cc7mQbke.o(.text+0x139aa): undefined reference to `alloca'
> /var/tmp//cc7mQbke.o: In function `l311__ZNSt5__padIwSt11char_traitsIwEE6_S_padERSt8ios_basewPwPKwiib':
> /var/tmp//cc7mQbke.o(.text+0x14db8): undefined reference to `alloca'
> /var/tmp//cc7mQbke.o: In function `l195__ZNKSt9money_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_bRSt8ios_basewe':
> /var/tmp//cc7mQbke.o(.text+0x15180): undefined reference to `alloca'
> /var/tmp//cc7mQbke.o(.text+0x153a5): undefined reference to `alloca'
> /var/tmp//cc7mQbke.o(.text+0x15580): more undefined references to `alloca' follow
> collect2: ld returned 1 exit status
> gmake[4]: [Output/ConditionalExpr.cbe] Error 1 (ignored)

You need to look into this. Why isn't alloca being linked in? Where is
it on FreeBSD? This comment goes for all the other C++/EH tests that
failed for "No alloca".

On FreeBSD it's defined in stdlib.h:

/*
* The alloca() function can't be implemented in C, and on some
* platforms it can't be implemented at all as a callable function.
* The GNU C compiler provides a built-in alloca() which we can use;
* in all other cases, provide a prototype, mainly to pacify various
* incarnations of lint. On platforms where alloca() is not in libc,
* programs which use it will fail to link when compiled with non-GNU
* compilers.
*/
#if __GNUC__ >= 2 || defined(__INTEL_COMPILER)
#undef alloca /* some GNU bits try to get cute and define this on their own */
#define alloca(sz) __builtin_alloca(sz)
#elif defined(lint)
void *alloca(size_t);
#endif

23

> >
> > ===================== MultiSource/Applications/sgefa
> >
> sgefa is a known XFAIL. See the nightly test results over the last
> several months. Actually, you should compare your test results with the
> 1.3 release test results (on or about August 13th, 2004). Please only
> report deltas from there.

Where can I find these? I see the nightly reports, but they don't say
what's XFAIL much less /how/ they're expected to fail.

yeah, that's a current deficiency in the nightly test. You can slog
through the downloadable log file from a given nightly test to find out
the "how" but there's no indication of which is an XFAIL and which
isn't. you just have to "know" :slight_smile:

Reid.

> > ===================== MultiSource/Benchmarks/Olden/power
> >
> > Only cbe failed:
> >
> > 6c6
> > < TR=0.81, TI=0.16, P0=6397.86, Q0=1288.50
> > ---
> > > TR=0.81, TI=0.16, P0=6397.87, Q0=1288.50
> > 26c26
> > < TR=0.80, TI=0.16, P0=7565.44, Q0=1525.86
> > ---
> > > TR=0.80, TI=0.16, P0=7565.44, Q0=1525.85
> > 34c34
> > < TR=0.79, TI=0.16, P0=7725.99, Q0=1558.54
> > ---
> > > TR=0.79, TI=0.16, P0=7725.99, Q0=1558.53
> > 36c36
> > < TR=0.79, TI=0.16, P0=8052.56, Q0=1625.04
> > ---
> > > TR=0.79, TI=0.16, P0=8052.56, Q0=1625.05
> > 74c74
> > < TR=0.79, TI=0.16, P0=7894.33, Q0=1592.81
> > ---
> > > TR=0.79, TI=0.16, P0=7894.32, Q0=1592.81
>
> This works on Linux (it did last night) so not sure why FreeBSD gives
> these rounding errors. Can you investigate? Does the test use fpcmp?

It does:

/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-llc Output/power.llc
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" llc power
/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-cbe Output/power.cbe
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" cbe power
/usr/home/llvm/obj/../test/Programs/RunSafely.sh 500 /dev/null Output/power.out-jit /usr/home/llvm/obj/tools/Debug/lli -force-interpreter=false Output/power.llvm.bc
/usr/home/llvm/obj/../test/Programs/DiffOutput.sh "/usr/home/llvm/obj/tools/Debug/fpcmp -r 0.00001" jit power

Sounds like we have some FreeBSD bugs .. not sure how though. Can you
file it?

> > ===================== MultiSource/Benchmarks/FreeBench/mason
> >
> > All fail (including native) with error "Could not open datafile test.in"
>
> When's the last time you cvs updated llvm-test, wiped out your build
> directory, reconfigured, and rebuilt? There have been numerous
> structural changes to llvm-test in the last week. It seems to have
> settled down now and the above error is indicative of at least one of
> the changes that fixed problems.

Last night, after I installed gcc 3.4.2. I also did a full update.

Well, that's recent enough .. another bug I guess. Can you file it?

> >
> > ===================== SingleSource/Regression/C++/EH/Output/ConditionalExpr
> >
> > Only cbe failed with:
> >
> > gcc Output/ConditionalExpr.cbe.c -std=c99 -fno-strict-aliasing -O2 -o Output/ConditionalExpr.cbe
> > Output/ConditionalExpr.cbe.c:1629: warning: conflicting types for built-in function 'memset'
> > Output/ConditionalExpr.cbe.c: In function `l141__ZNKSt7collateIwE12do_transformEPKwS2_':
> > Output/ConditionalExpr.cbe.c:29424: warning: implicit declaration of function `alloca'
> > /var/tmp//cc7mQbke.o: In function `l292__ZNKSt8time_getIwSt19istreambuf_iteratorIwSt11char_traitsIwEEE15_M_extract_nameERS3_S5_RiPPKwjRSt12_Ios_Iostate':
> > /var/tmp//cc7mQbke.o(.text+0x122e3): undefined reference to `alloca'
> > /var/tmp//cc7mQbke.o: In function `l164__ZNKSt8time_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_RSt8ios_basewPK2tmcc':
> > /var/tmp//cc7mQbke.o(.text+0x139aa): undefined reference to `alloca'
> > /var/tmp//cc7mQbke.o: In function `l311__ZNSt5__padIwSt11char_traitsIwEE6_S_padERSt8ios_basewPwPKwiib':
> > /var/tmp//cc7mQbke.o(.text+0x14db8): undefined reference to `alloca'
> > /var/tmp//cc7mQbke.o: In function `l195__ZNKSt9money_putIwSt19ostreambuf_iteratorIwSt11char_traitsIwEEE6do_putES3_bRSt8ios_basewe':
> > /var/tmp//cc7mQbke.o(.text+0x15180): undefined reference to `alloca'
> > /var/tmp//cc7mQbke.o(.text+0x153a5): undefined reference to `alloca'
> > /var/tmp//cc7mQbke.o(.text+0x15580): more undefined references to `alloca' follow
> > collect2: ld returned 1 exit status
> > gmake[4]: [Output/ConditionalExpr.cbe] Error 1 (ignored)
>
> You need to look into this. Why isn't alloca being linked in? Where is
> it on FreeBSD? This comment goes for all the other C++/EH tests that
> failed for "No alloca".

On FreeBSD it's defined in stdlib.h:

/*
* The alloca() function can't be implemented in C, and on some
* platforms it can't be implemented at all as a callable function.
* The GNU C compiler provides a built-in alloca() which we can use;
* in all other cases, provide a prototype, mainly to pacify various
* incarnations of lint. On platforms where alloca() is not in libc,
* programs which use it will fail to link when compiled with non-GNU
* compilers.
*/
#if __GNUC__ >= 2 || defined(__INTEL_COMPILER)
#undef alloca /* some GNU bits try to get cute and define this on their own */
#define alloca(sz) __builtin_alloca(sz)
#elif defined(lint)
void *alloca(size_t);
#endif

Okay, so the question is, why doesn't the builtin get recognized. I
guess you could file this as a bug too.

Make sure to mark all these bugs as specific to FreeBSD as they don't
happen on Linux/Solaris.

Thanks Jeff,

Reid.