CodeGen runtime tests

Hi LLVMdev,

I have a test that checks if CodeGen implements shift instruction for illegal types correctly. The test itself checks the runtime result of the operation using lli tool. This runtime check approach is quite different from tests in test/CodeGen, as they use llc tool mainly.

Should my test be places somewhere else? Or should it be reimplemented?
http://reviews.llvm.org/D7752

  • Paweł

Hi LLVMdev,

I have a test that checks if CodeGen implements shift instruction for
illegal types correctly. The test itself checks the runtime result of the
operation using lli tool. This runtime check approach is quite different
from tests in test/CodeGen, as they use llc tool mainly.

Why are you testing it that way, rather than the way other
test/CodeGen tests work - verifying assembly?

The regression suite generally (with some exceptions, like the JIT)
doesn't execute newly-compiled code. We just test that we produce the
right code, not that it executes correctly. (we do this for speed,
isolation/stability, etc) We do have the "nightly test suite" (in the
test-suite repository) that handles executing arbitrary code examples,
etc - but that's mostly for large-ish programs with standalone
functionality, not targeted tests.

Hi LLVMdev,

I have a test that checks if CodeGen implements shift instruction for
illegal types correctly. The test itself checks the runtime result of the
operation using lli tool. This runtime check approach is quite different
from tests in test/CodeGen, as they use llc tool mainly.

Why are you testing it that way, rather than the way other
test/CodeGen tests work - verifying assembly?

Before the bug was fixed I haven’t known what assembly to expect. I can now write a test that checks the assembly but the test will be both target and implementation specific. Do you think it is better?

> Hi LLVMdev,
>
> I have a test that checks if CodeGen implements shift instruction for
> illegal types correctly. The test itself checks the runtime result of
> the
> operation using lli tool. This runtime check approach is quite different
> from tests in test/CodeGen, as they use llc tool mainly.

Why are you testing it that way, rather than the way other
test/CodeGen tests work - verifying assembly?

Before the bug was fixed I haven't known what assembly to expect. I can now
write a test that checks the assembly but the test will be both target and
implementation specific. Do you think it is better?

generally, yes (yeah, it's not perfect & everything has tradeoffs -
but that's generally the kind of testing that's preferred)

- David