JIT tests on AArch64

Hi all,

I am seeing a number of JIT unit test failures on AArch64. Some googling returns
past discussions (circa 2014) but as far as I can tell, everything should be working.
I listed the failures below.

These are from a Release build. A Debug build does not exhibit the failures. I built
with gcc 6.1.0 on SuSE 12. Switching to gcc 8.2.0 also makes the problem go away,
both in Debug and Release builds.

Is there a known problem with gcc 6.1.0 on AArch64? I considered the possiblity
that the tests (or libraries) somehow wander into undefined behavior territory
but my guess is that's unlikely.

                                                            -David

+lhames, JIT Ring Master

David - do you have any further info on the failures? How are these tests failing (what errors, crashes, etc? backtraces?)? Might get something useful out of running them under a sanitizer or valgrind (might not - not sure if all of their lower-level behavior is understood/valid with those kinds of checkers)

Just a quick note to say that I saw this. I'm tracking down what looks
like another issue which may be undefined behavior within a PassManager
test. That's more critical for us at the moment so I'm going to
diagnose and send a report about that before getting back to the JIT
tests. In the meantime, if Lang knows anything, that would be helpful.

                        -David

David Blaikie via llvm-dev <llvm-dev@lists.llvm.org> writes:

Hi David,

I am afraid nothing leaps to mind. Once you have a chance to look at them again please let me know – I’d be happy to look at backtraces / failures.

Cheers,
Lang.

Lang Hames via llvm-dev <llvm-dev@lists.llvm.org> writes:

I am afraid nothing leaps to mind. Once you have a chance to look at
them again please let me know -- I'd be happy to look at backtraces /
failures.

Huh. Now the problem has disappeared. A colleague updated to ToT and
now I'm not seeing the failures, even with gcc 6.1.0 in Release mode.

Most disconcerting...

I've attached valgrind output from one of the failures I saw last week.
This and the PassManager issue I mentioned last week seem to be related
to the use of lambdas. I thought it might be related to this but it is
not, as using the workaround flags did not fix the problem:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77686

                            -David

valgrind.release.out (46.3 KB)

"David A. Greene via llvm-dev" <llvm-dev@lists.llvm.org> writes:

Lang Hames via llvm-dev <llvm-dev@lists.llvm.org> writes:

I am afraid nothing leaps to mind. Once you have a chance to look at
them again please let me know -- I'd be happy to look at backtraces /
failures.

Huh. Now the problem has disappeared. A colleague updated to ToT and
now I'm not seeing the failures, even with gcc 6.1.0 in Release mode.

User error. I was building on the wrong branch!

I am still seeing these errors with gcc 6.1.0 in Release mode.

Is there anything else useful I can provide to help track this down?

When I tried gcc 8.2.0 these tests passed but some others broke. :frowning:

                         -David

Hi David,

I don’t have access to a gcc 6.1.0 / aarch64 setup, but I’ve tried the tests locally and they’re passing asan/ubsan/tsan clean, and there is no obvious uninitialized memory use in the test case (though who knows where it might be inlined from in a release build).

When you say other tests broke with 8.2.0, do you mean other ORC tests specifically, or just other LLVM tests?

Cheers,
Lang.

Other general LLVM tests fail with 8.2.0.

Don't spend too much time chasing this down. I just wanted to see if anyone is aware of issues with gcc 6.1.0. Sounds like not. I will keep investigating on our end. It may be something with our environment is strange. If/when I figure something out I'll report it.

                                         -David

Ok.

I am not aware of any specific issues with gcc 6.1.0, but for what it’s worth ORC code is heavy on lambdas, std::function, and promises/futures compared to the rest of LLVM. In the past it has ICE’d a few GCC versions (workarounds have been made in tree for this) and shown up bugs in MSVC’s standard library implementation. Bug reports for those features are the first place I look now when I see a failing builder running an older GCC fail while the rest of the builders are green.

Cheers,
Lang.