Disabling assertions in llvm.org releases

Currently LLVM releases, as downloaded from llvm.org, have assertions enabled.
(I'm told Apple ships LLVM with assertions disabled, which is why I mention
llvm.org here). What do people think of disabling assertions?

The advantage of disabling assertions is that LLVM runs faster. I don't know
how much faster. It also makes for fairer comparisons with gcc releases, which
have all but the cheapest checks disabled.

The disadvantage of disabling assertions is that we get less information in bug
reports.

An additional consideration here is clang: while LLVM itself is reasonably
mature, that's not the case for clang. I don't know if it is practical to
ship LLVM with assertions disabled, and clang with assertions enabled.

A different possibility is to analyse which checks are the most expensive,
and just disable those.

Thoughts?

Ciao,

Duncan.

I am running with assertions enabled and it was very useful in finding
bugs on platforms I can't (or rarely) test on.
Had assertions been off I think bad/invalid code would have been
generated silently, resulting in a much harder to debug segfault.

There are certain assertions that shouldn't be disabled IMHO.
Like those in codegen that check for "impossible" conditions, or
conditions under which codegen can't continue.

For example the assertions about relocations being in bounds on PowerPC.
Perhaps these shouldn't be assertions at all, but there are far too many
asserts to go over them one-by-one now and decide on converting to
llvm_error.

Maybe we should convert the "cheap" (invoked once per instruction at
most) assert checks to something else, and then you could turn off the
rest of the assertions.

Best regards,
--Edwin

Török Edwin schrieb:

Had assertions been off I think bad/invalid code would have been
generated silently, resulting in a much harder to debug segfault.

Just a thought: how about distributing a "production build" with only the most important assertions active, and a "diagnostic build" that has all assertion checking on?
In that case, if there's "funny behaviour" in the generated code, people can be instructed to try running their code through diagnostic build and see whether it assertion checks.
Or they could be told to use the diagnostic LLVM when building production code just to be sure, but normally, the standard build will suffice.

Regards,
Jo

I strongly suggest not to.

LLVM's assertions won't only get LLVM's own bugs, but also Clang bugs.
IR is not perfect and sometimes allow wrong code to be produced and
validated, only to break at codegen pass on an assert. As they're a
last resort in finding problems, I fear that if we turn them off, bad
code will be generated and, as they'll be running on the user, it'll
be very difficult to trace or even find out that there is a bug in the
compiler in the first place.

Maybe what we need is a cheaper error check. Less computation on the
checking and more on the reporting, so only if there is an error, the
red button is pressed.

So, instead of:

assert(condition && "error message");

we do:

if (!condition) error("message");

I don't think it's that much difference, but that's the cheaper I can
think of...

cheers,
--renato

PS: asserts get inlined normally, don't they?

Userland don't tend to have source trees to build with diagnostic
flags. Maybe if distributions had two packages (prod / diag) that
could make it easier.

Run time checks are no good, too. The similar of a -vvv flag would
avoid multiple versions of the compiler installed, but that also has
performance issues, as you always have the "if (diag_level > X)"
before each assert.

As always, a trade off.

Btw, has anyone measured the delay of full assertions compared to no
assertions at all? We may be discussing over nothing...

cheers,
--renato

Hi Torok,

I am running with assertions enabled and it was very useful in finding
bugs on platforms I can't (or rarely) test on.
Had assertions been off I think bad/invalid code would have been
generated silently, resulting in a much harder to debug segfault.

are you using the binaries supplied by llvm.org, or building yourself? There
is a big difference between what is useful for developers (i.e. helpful back
traces etc) and what is useful for users (i.e. speed). Since developers
typically build LLVM themselves, they can always turn assertions on.

Ciao,

Duncan.

Right now I am building it myself, but for ClamAV 0.97 I intend to
make it work with the official LLVM 2.8 release (from llvm.org, or
provided via a package in the distribution).

Best regards,
--Edwin

Hi Renato,

Btw, has anyone measured the delay of full assertions compared to no
assertions at all? We may be discussing over nothing...

I measured the effect of turning assertions off when compiling a huge file
(gcc.c from http://people.csail.mit.edu/smcc/projects/single-file-programs/)
using dragonegg, and when using opt and llc on the unoptimized bitcode:

  * gcc-4.5+dragonegg -S -O3 -> 22% faster with assertions off
  * opt -std-compile-opts -> 22% faster with assertions off
  * llc -> 16% faster with assertions off

So turning assertions off seems to give a significant speed-up.

Ciao,

Duncan.

Hi Duncan,

Yes, the numbers speak for themselves. :wink:

cheers,
--renato