New benchmark in test-suite

Hi Daniel,

I'm trying to add LivermoreLoops test to the benchmark suite (tar ball
attached), but I'm getting the error below:

--- Tested: 2 tests --
FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.compile_time (1 of 2)
FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.execution_time (2 of 2)

When I use the option to only run this test:

--only-test SingleSource/Benchmarks/LivermoreLoops

I can see the binary and if I execute, it runs (still missing bits and
bobs, but ignore that for now). I've seen some files with extension
"reference_output", does that have anything to do with it?

Since it "fails" on "compile_time", I'm guessing I'm not quite there
yet, though.

If you want to reproduce it, just add "LivermoreLoops" to the
Benchmarks' Makefile and untar the attached to it.

LivermoreLoops.tar.gz (41 KB)

Hey Renato,

You are right, the failure on compile_time indicates that the test isn’t even building. As provided, the tests don’t actually define the cpuida() or calculateMHz() functions so that seems expected to me.

The compile failures end up getting buried in the logs, but they will either be in the test.log file in the top-level sandbox directory, or inside the .llvm.o.compile files in the Output directory for that test (i.e. SingleSource/Benchmarks/LivermoreLoops/Output/…).

  • Daniel

You are right, the failure on compile_time indicates that the test isn't
even building. As provided, the tests don't actually define the cpuida() or
calculateMHz() functions so that seems expected to me.

I defined both functions as NOPs.

I got what it was. The original makefile had a "-o lloops" on the
compile command and I wrongfully copied it to the LDFLAGS, so it was
generating the binary on the current dir, not on the Output dir, where
it should belong. :slight_smile:

The compile failures end up getting buried in the logs, but they will either
be in the test.log file in the top-level sandbox directory, or inside the
<filename>.llvm.o.compile files in the Output directory for that test (i.e.
SingleSource/Benchmarks/LivermoreLoops/Output/...).

Yes, I saw those logs and they were fine (3 warnings, 0 errors), so I
got confused. Is there a place where you log the verbose Makefile
output? That would have given me a better clue as to what was
happening.

cheers,
--renato

> You are right, the failure on compile_time indicates that the test isn't
> even building. As provided, the tests don't actually define the cpuida()
or
> calculateMHz() functions so that seems expected to me.

I defined both functions as NOPs.

I got what it was. The original makefile had a "-o lloops" on the
compile command and I wrongfully copied it to the LDFLAGS, so it was
generating the binary on the current dir, not on the Output dir, where
it should belong. :slight_smile:

> The compile failures end up getting buried in the logs, but they will
either
> be in the test.log file in the top-level sandbox directory, or inside the
> <filename>.llvm.o.compile files in the Output directory for that test
(i.e.
> SingleSource/Benchmarks/LivermoreLoops/Output/...).

Yes, I saw those logs and they were fine (3 warnings, 0 errors), so I
got confused. Is there a place where you log the verbose Makefile
output? That would have given me a better clue as to what was
happening.

The test.log file at the top-level of the sandbox should have it.

- Daniel