Adding Polybench to the test suite

Hi all,

We are trying to add Polybench[1] to the LLVM test suite. So far we
have all but five benchmarks working fine. The problems we're having
in those five benchmarks are due to gcc's versus clang's different
ways to output 'Nan's. We are working to solve this.

Although the patch is not ready yet to be included in the LLVM test
suite, we would like to have some feedback from the LLVM comunity to
know if we need to change anything else. I've removed the problematic
benchmarks from the Makefiles for now, so you should be able to run
all of the benchmarks, and all of them should pass the tests.

[1] http://www.cse.ohio-state.edu/~pouchet/software/polybench/

Cheers,

Douglas

Polybench.zip (107 KB)

Hi Douglas,

We are trying to add Polybench[1] to the LLVM test suite. So far we
have all but five benchmarks working fine. The problems we're having
in those five benchmarks are due to gcc's versus clang's different
ways to output 'Nan's. We are working to solve this.

is it normal (i.e. valid) that they output NaN's differently, or is it a
bug?

Ciao, Duncan.

Hi Duncan,

Hi Douglas,

We are trying to add Polybench[1] to the LLVM test suite. So far we
have all but five benchmarks working fine. The problems we're having
in those five benchmarks are due to gcc's versus clang's different
ways to output 'Nan's. We are working to solve this.

is it normal (i.e. valid) that they output NaN's differently, or is it a
bug?

We get basically the same output with clang and gcc. The only
difference is that we get a lot of '-nan's using gcc and we get 'nan's
using clang. Tobias sent me this earlier today:
"An implementation may give zero and values that are not
floating-point numbers (such as infinities and NaNs) a sign or may
leave them unsigned. Wherever such values are unsigned, any
requirement in this International Standard to retrieve the sign shall
produce an unspecified sign, and any requirement to set the sign
shall be ignored."

So I don't think it is bug, but we're getting different results, anyway.

Cheers,

Douglas

From: llvmdev-bounces@cs.uiuc.edu [mailto:llvmdev-bounces@cs.uiuc.edu]
On Behalf Of Douglas Teixeira
Sent: Wednesday, May 30, 2012 4:51 PM
To: Duncan Sands
Cc: Tobias Grosser; llvmdev@cs.uiuc.edu
Subject: Re: [LLVMdev] Adding Polybench to the test suite

Hi Duncan,

> Hi Douglas,
>
>> We are trying to add Polybench[1] to the LLVM test suite. So far we
>> have all but five benchmarks working fine. The problems we're having
>> in those five benchmarks are due to gcc's versus clang's different
>> ways to output 'Nan's. We are working to solve this.
>
> is it normal (i.e. valid) that they output NaN's differently, or is it
> a bug?
>

We get basically the same output with clang and gcc. The only difference is
that we get a lot of '-nan's using gcc and we get 'nan's using clang. Tobias sent
me this earlier today:
"An implementation may give zero and values that are not floating-point
numbers (such as infinities and NaNs) a sign or may
leave them unsigned. Wherever such values are unsigned, any
requirement in this International Standard to retrieve the sign shall produce
an unspecified sign, and any requirement to set the sign shall be ignored."

So I don't think it is bug, but we're getting different results, anyway.

Cheers,

Douglas

This seems strange. 0 and -0 have different meanings, and an unsigned -0 doesn't make much sense, to me anyway.

That said, depending on the GCC version I found that it produced strange bugs with 0 and NaN results at anything but -O0, and had to set it to that to match llvm-gcc's (correct) output in the test suites for 2.8 when using GCC 4.2 as the reference. That might be worth trying. Oddly the manual for that version claimed that no unsafe floating point optimizations would be enabled at any -O level.

Cheers,
Gordon