OpenJDK8 failed to work after compiled by LLVM 8 for X86

Leslie_Zhai2 · September 10, 2018, 4:20pm

Hi all,

OpenJDK8 jdk8u-dev[1] is just able to work after compiled with LLVM 3.9.1 for X86:

$ ./build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/bin/java -version
openjdk version "1.8.0-internal-debug"
OpenJDK Runtime Environment (build 1.8.0-internal-debug-xiangzhai_2018_09_09_21_08-b00)
OpenJDK 64-Bit Server VM (build 25.71-b00-debug, mixed mode)

$ strings ./build/linux-x86_64-normal-server-slowdebug/images/j2sdk-image/bin/java

grep clang

clang version 3.9.1 (tags/RELEASE_391/final)

But it failed to work after compiled with LLVM 8 for X86:

$ ./build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/bin/java -version
Segmentation fault

$ gdb --ex run --args ./build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/bin/java -version
GNU gdb (GDB) Fedora 7.12.1-48.fc25
...
Starting program: /home/xiangzhai/project/jdk8u-llvm/build/linux-x86_64-normal-server-fastdebug/images/j2sdk-image/bin/java -version
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
[ Legend: Modified register | Code | Heap | Stack | String ]
───────────────────────────────────────────────────────────────────────────────────────────────[ registers ]────
$rax : 0x0
$rbx : 0x0
$rcx : 0x7ffff7315eb0 → 0x00007ffff6ae30a0 → <NativeCallStack::print_on(outputStream*)+0> xor edx, edx
$rdx : 0x0
$rsp : 0x7fffffff9328 → 0x00007ffff6a76822 → <MemTracker::init_tracking_level()+178> mov edi, ebx
$rbp : 0x7fffffff93c0 → 0x00007fffffff94f0 → 0x00007fffffff9510 → 0x0000000000000002
$rsi : 0x0
$rdi : 0x7ffff7315e70 → 0x00007ffff7315eb0 → 0x00007ffff6ae30a0 → <NativeCallStack::print_on(outputStream*)+0> xor edx, edx
$rip : 0x7ffff6ae2f5b → <NativeCallStack::NativeCallStack(int,+0> mov QWORD PTR [rdi], rcx
$r8 : 0x1
$r9 : 0xf
$r10 : 0x64
$r11 : 0x0
$r12 : 0x7fffffffdc88 → 0x00007fffffffe04c → "/home/xiangzhai/project/jdk8u-llvm/build/linux-x86[...]"
$r13 : 0x7fffffffdca0 → 0x00007fffffffe0bf → "XDG_VTNR=1"
$r14 : 0x7fffffff9330 → "NMT_LEVEL_10535"
$r15 : 0x6
$eflags: [carry parity adjust zero sign trap INTERRUPT direction overflow RESUME virtualx86 identification]
$ss: 0x002b $ds: 0x0000 $cs: 0x0033 $es: 0x0000 $gs: 0x0000 $fs: 0x0000
───────────────────────────────────────────────────────────────────────────────────────────────────[ stack ]────
0x00007fffffff9328│+0x00: 0x00007ffff6a76822 → <MemTracker::init_tracking_level()+178> mov edi, ebx ← $rsp
0x00007fffffff9330│+0x08: "NMT_LEVEL_10535" ← $r14
0x00007fffffff9338│+0x10: 0x0035333530315f4c ("L_10535"?)
0x00007fffffff9340│+0x18: 0x0000000000000001
0x00007fffffff9348│+0x20: 0x0000000000602190 → 0x00007ffff5eb7000 → 0x00010102464c457f
0x00007fffffff9350│+0x28: 0x0000000000000004
0x00007fffffff9358│+0x30: 0x000000066474e551
0x00007fffffff9360│+0x38: 0x0000000000000000
────────────────────────────────────────────────────────────────────────────────────────[ code:i386:x86-64 ]────
0x7ffff6ae2f4b <print_owned_locks_on_error(outputStream*)+187> add eax, 0x1f0f00
0x7ffff6ae2f50 <NativeCallStack::NativeCallStack(int,+0> mov rcx, QWORD PTR [rip+0x842781] # 0x7ffff73256d8
0x7ffff6ae2f57 <NativeCallStack::NativeCallStack(int,+0> add rcx, 0x10
→ 0x7ffff6ae2f5b <NativeCallStack::NativeCallStack(int,+0> mov QWORD PTR [rdi], rcx
0x7ffff6ae2f5e <NativeCallStack::NativeCallStack(int,+0> mov DWORD PTR [rdi+0x28], 0x0
0x7ffff6ae2f65 <NativeCallStack::NativeCallStack(int,+0> add rdi, 0x8
0x7ffff6ae2f69 <NativeCallStack::NativeCallStack(int,+0> test edx, edx
0x7ffff6ae2f6b <NativeCallStack::NativeCallStack(int,+0> je 0x7ffff6ae2f7b <NativeCallStack::NativeCallStack(int, bool)+43>
0x7ffff6ae2f6d <NativeCallStack::NativeCallStack(int,+0> mov eax, esi
─────────[ source:/home/xiangzhai/project/jdk8u-llvm/hotspot/src/share/vm/utilities/nativeCallStack.cpp+33 ]────
28 #include "utilities/nativeCallStack.hpp"
29
30 const NativeCallStack NativeCallStack::EMPTY_STACK(0, false);
31
32 NativeCallStack::NativeCallStack(int toSkip, bool fillStack) :
→ 33 _hash_value(0) {
34
35 #if !PLATFORM_NATIVE_STACK_WALKING_SUPPORTED
36 fillStack = false;
37 #endif
38
─────────────────────────────────────────────────────────────────────────────────────────────────[ threads ]────
[#0] Id 1, Name: "java", stopped, reason: SIGSEGV
───────────────────────────────────────────────────────────────────────────────────────────────────[ trace ]────
[#0] 0x7ffff6ae2f5b → Name: NativeCallStack::NativeCallStack(this=0x7ffff7315e70 <NativeCallStack::EMPTY_STACK>, toSkip=0x0, fillStack=0x0)
[#1] 0x7ffff6a76822 → Name: MemTracker::init_tracking_level()
[#2] 0x7ffff62eff49 → Name: MemTracker::tracking_level()
[#3] 0x7ffff62eff49 → Name: ResourceObj::operator new(size=0x20, type=ResourceObj::STACK_OR_EMBEDDED, flags=<optimized out>)
[#4] 0x7ffff61369ec → Name: _GLOBAL__sub_I_c1_Greedy.cpp()
[#5] 0x7ffff7de7d9a → Name: call_init(l=<optimized out>, argc=0x2, argv=0x7fffffffdc88, env=0x7fffffffdca0)
[#6] 0x7ffff7de7eab → Name: call_init(env=0x7fffffffdca0, argv=0x7fffffffdc88, argc=0x2, l=<optimized out>)
[#7] 0x7ffff7de7eab → Name: _dl_init(main_map=0x602190, argc=0x2, argv=0x7fffffffdc88, env=0x7fffffffdca0)
[#8] 0x7ffff7dece46 → Name: dl_open_worker(a=0x7fffffff97e0)
[#9] 0x7ffff7de7c44 → Name: _dl_catch_error(objname=0x7fffffff97d0, errstring=0x7fffffff97d8, mallocedp=0x7fffffff97cf, operate=0x7ffff7decae0 <dl_open_worker>, args=0x7fffffff97e0)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────
0x00007ffff6ae2f5b in NativeCallStack::NativeCallStack (this=0x7ffff7315e70 <NativeCallStack::EMPTY_STACK>, toSkip=0x0, fillStack=0x0) at /home/xiangzhai/project/jdk8u-llvm/hotspot/src/share/vm/utilities/nativeCallStack.cpp:33
33 _hash_value(0) {
gef➤ c
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
gef➤

----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< ---

I started to use and contribute clang static analyzer[2] just for finding the bugs for open source software from LLVM 3.x, perhaps it is a bug of LLVM 8, because jdk8u is just able to print out '-version' after compiled with llvm-3.9.1, I will rebuild the LLVM 8 again, then rebuild jdk8u for triple check.

1. Workaround-compile-with-llvm.patch https://raw.githubusercontent.com/xiangzhai/jdk8u-dev/master/Workaround-compile-with-llvm.patch

2. [analyzer] Teach the MallocChecker about Glib API for two arguments · llvm-mirror/clang@a530e82 · GitHub

3. The build option of LLVM 8 (bootstrap with LLVM 3.9.1):

cmake .. -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_C_FLAGS="-fPIC" \
-DCMAKE_CXX_FLAGS="-std=c++11 -fPIC" \
-DLLVM_BUILD_LLVM_DYLIB=ON \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_ENABLE_EH=ON \
-DLLVM_BUILD_TESTS=ON \
-DLLVM_BUILD_DOCS=OFF \
-DLLVM_ENABLE_SPHINX=OFF \
-DLLVM_ENABLE_DOXYGEN=OFF \
-DLLDB_DISABLE_LIBEDIT=1 \
-DSPHINX_WARNINGS_AS_ERRORS=OFF \
-DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
-DFFI_LIBRARY_DIR:PATH="$(pkg-config --variable=libdir libffi)" \
-DLLVM_BINUTILS_INCDIR=/usr/include \
-DLLVM_LIBDIR_SUFFIX=64 \
-DLIBCXX_ENABLE_ABI_LINKER_SCRIPT=ON \
-DLIBUNWIND_ENABLE_SHARED=ON \
-DLIBCXXABI_USE_LLVM_UNWINDER=ON \
-DLLDB_TEST_C_COMPILER=clang \
-DLLDB_TEST_CXX_COMPILER=clang++ \
-DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-redhat-linux" \
-DCLANG_VENDOR="LLVM China"

4. $ clang -v
LLVM China clang version 8.0.0 (git@github.com:llvm-mirror/clang.git 81ef98628ebf5186d746c0986dcbf5073e842043) (git@github.com:llvm-mirror/llvm.git e1aac9723d55497e74d83d216329f08d9842e494) (based on LLVM 8.0.0svn)
Target: x86_64-redhat-linux
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-redhat-linux/6.4.1
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/6.4.1
Found candidate GCC installation: /usr/lib/gcc/i686-redhat-linux/6.4.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/6.4.1
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/6.4.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64

DimitryAndric · September 10, 2018, 4:59pm

Hi Leslie,

This is likely the same problem as was reported in 225054 – java/openjdk* : fails to build with clang 6.0 (blocks 571 ports), and fixed by the following patch:

https://svnweb.freebsd.org/ports/head/java/openjdk8/files/patch-hotspot_src_share_vm_services_memTracker.cpp?view=markup&pathrev=459368

Can you please try that out, and see if it fixes it for you too?

-Dimitry

Leslie_Zhai2 · September 11, 2018, 1:38am

Hi Dimitry,

Thanks for your kind response!

Thanks for the commit message of Jung's patch, I found that the bug had been fixed in OpenJDK 12 by Zhengyu [JDK-8205965] SIGSEGV on write to NativeCallStack::EMPTY_STACK - Java Bug System But only backported to 11. So Jung could backport it for OpenJDK 8, thanks a lot!

But I argue that the root cause might be in the compiler side, why clang-3.9.1, gcc-6.4.1 couldn't reproduce the bug? And it might be work for clang-4.0 [JDK-8208494] Compilation errors with clang-4.0 - Java Bug System I will test LLVM 5 to check out whether or not reproducible.

DimitryAndric · September 11, 2018, 8:55am

Hi Leslie,

The problem really lies in the OpenJDK code, as it is attempting to
write to a const object. If this seems to work with certain compiler(s)
and optimization settings, it is just luck.

Here is a reduced example, which shows the issue:

Leslie_Zhai2 · September 11, 2018, 4:09pm

Hi Dimitry,

Thank you so much for the reduced testcase! It is able to reproduce after compiled with clang-8 optimized for X86:

$ clang++ -O3 -S -c JDK-8205969.cpp -o JDK-8205969-opt-8.0.s
$ clang++ -O3 -c JDK-8205969.cpp -o JDK-8205969-opt-8.0.o
$ clang++ -o JDK-8205969-opt-8.0.out JDK-8205969-opt-8.0.o

$ ./JDK-8205969-opt-8.0.out
Segmentation fault

$ cat JDK-8205969-opt-8.0.s
.text
.file "JDK-8205969.cpp"
.globl main # -- Begin function main
.p2align 4, 0x90
.type main,@function
main: # @main
.cfi_startproc
# %bb.0:
xorps %xmm0, %xmm0
movups %xmm0, _ZN15NativeCallStack11EMPTY_STACKE+16(%rip)
movups %xmm0, _ZN15NativeCallStack11EMPTY_STACKE(%rip)
xorl %eax, %eax
retq
.Lfunc_end0:
.size main, .Lfunc_end0-main
.cfi_endproc
# -- End function
.type _ZN15NativeCallStack11EMPTY_STACKE,@object # @_ZN15NativeCallStack11EMPTY_STACKE
.section .rodata,"a",@progbits

                          ^\-\-\- READ\-ONLY segment

     \.globl  \_ZN15NativeCallStack11EMPTY\_STACKE
     \.p2align        3

_ZN15NativeCallStack11EMPTY_STACKE:
.zero 32
.size _ZN15NativeCallStack11EMPTY_STACKE, 32

     \.ident  &quot;LLVM China clang version 8\.0\.0 \(git@github\.com:llvm\-mirror/clang\.git 81ef98628ebf5186d746c0986dcbf5073e842043\) \(git@github\.com:llvm\-mirror/llvm\.git e1aac9723d55497e74d83d216329f08d9842e494\) \(based on LLVM 8\.0\.0svn\)&quot;
     \.section        &quot;\.note\.GNU\-stack&quot;,&quot;&quot;,@progbits
     \.addrsig

But it is *not* able to reproduce with clang-8 not optimized for X86:

$ clang++ -O0 -S -c JDK-8205969.cpp -o JDK-8205969-unopt-8.0.s
$ clang++ -O0 -c JDK-8205969.cpp -o JDK-8205969-unopt-8.0.o
$ clang++ -o JDK-8205969-unopt-8.0.out JDK-8205969-unopt-8.0.o

$ ./JDK-8205969-unopt-8.0.out

No segfault

$ cat JDK-8205969-unopt-8.0.s
.text
.file "JDK-8205969.cpp"
.section .text.startup,"ax",@progbits
.p2align 4, 0x90 # -- Begin function __cxx_global_var_init
.type __cxx_global_var_init,@function
__cxx_global_var_init: # @__cxx_global_var_init
.cfi_startproc
# %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
movabsq $_ZN15NativeCallStack11EMPTY_STACKE, %rdi
callq _ZN15NativeCallStackC2Ev
popq %rbp
.cfi_def_cfa %rsp, 8
retq
.Lfunc_end0:
.size __cxx_global_var_init, .Lfunc_end0-__cxx_global_var_init
.cfi_endproc
# -- End function
.section .text._ZN15NativeCallStackC2Ev,"axG",@progbits,_ZN15NativeCallStackC2Ev,comdat
.weak _ZN15NativeCallStackC2Ev # -- Begin function _ZN15NativeCallStackC2Ev
.p2align 4, 0x90
.type _ZN15NativeCallStackC2Ev,@function
_ZN15NativeCallStackC2Ev: # @_ZN15NativeCallStackC2Ev
.cfi_startproc
# %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
movq %rdi, -8(%rbp)
movq -8(%rbp), %rdi
movl $0, -12(%rbp)
movq %rdi, -24(%rbp) # 8-byte Spill
.LBB1_1: # =>This Inner Loop Header: Depth=1
cmpl $4, -12(%rbp)
jge .LBB1_4
# %bb.2: # in Loop: Header=BB1_1 Depth=1
movslq -12(%rbp), %rax
movq -24(%rbp), %rcx # 8-byte Reload
movq $0, (%rcx,%rax,8)
# %bb.3: # in Loop: Header=BB1_1 Depth=1
movl -12(%rbp), %eax
addl $1, %eax
movl %eax, -12(%rbp)
jmp .LBB1_1
.LBB1_4:
popq %rbp
.cfi_def_cfa %rsp, 8
retq
.Lfunc_end1:
.size _ZN15NativeCallStackC2Ev, .Lfunc_end1-_ZN15NativeCallStackC2Ev
.cfi_endproc
# -- End function
.text
.globl main # -- Begin function main
.p2align 4, 0x90
.type main,@function
main: # @main
.cfi_startproc
# %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
subq $16, %rsp
movl $0, -4(%rbp)
movabsq $_ZN15NativeCallStack11EMPTY_STACKE, %rdi
callq _ZN15NativeCallStackC2Ev
xorl %eax, %eax
addq $16, %rsp
popq %rbp
.cfi_def_cfa %rsp, 8
retq
.Lfunc_end2:
.size main, .Lfunc_end2-main
.cfi_endproc
# -- End function
.section .text.startup,"ax",@progbits
.p2align 4, 0x90 # -- Begin function _GLOBAL__sub_I_JDK_8205969.cpp
.type _GLOBAL__sub_I_JDK_8205969.cpp,@function
_GLOBAL__sub_I_JDK_8205969.cpp: # @_GLOBAL__sub_I_JDK_8205969.cpp
.cfi_startproc
# %bb.0:
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
callq __cxx_global_var_init
popq %rbp
.cfi_def_cfa %rsp, 8
retq
.Lfunc_end3:
.size _GLOBAL__sub_I_JDK_8205969.cpp, .Lfunc_end3-_GLOBAL__sub_I_JDK_8205969.cpp
.cfi_endproc
# -- End function
.type _ZN15NativeCallStack11EMPTY_STACKE,@object # @_ZN15NativeCallStack11EMPTY_STACKE
.bss

     ^\-\-\- R/W segment

     \.globl  \_ZN15NativeCallStack11EMPTY\_STACKE
     \.p2align        3

_ZN15NativeCallStack11EMPTY_STACKE:
.zero 32
.size _ZN15NativeCallStack11EMPTY_STACKE, 32

     \.section        \.init\_array,&quot;aw&quot;,@init\_array
     \.p2align        3
     \.quad   \_GLOBAL\_\_sub\_I\_JDK\_8205969\.cpp

     \.ident  &quot;LLVM China clang version 8\.0\.0 \(git@github\.com:llvm\-mirror/clang\.git 81ef98628ebf5186d746c0986dcbf5073e842043\) \(git@github\.com:llvm\-mirror/llvm\.git e1aac9723d55497e74d83d216329f08d9842e494\) \(based on LLVM 8\.0\.0svn\)&quot;
     \.section        &quot;\.note\.GNU\-stack&quot;,&quot;&quot;,@progbits
     \.addrsig
     \.addrsig\_sym \_\_cxx\_global\_var\_init
     \.addrsig\_sym \_GLOBAL\_\_sub\_I\_JDK\_8205969\.cpp
     \.addrsig\_sym \_ZN15NativeCallStack11EMPTY\_STACKE

----- 8< -------- 8< -------- 8< -------- 8< -------- 8< -------- 8< ---

Furthermore clang-3.9.1 *optimized* for X86 is *not* able to reproduce the issue neither. It behaviors like gcc-6.4.1 and gcc-8. But I will also check out clang-8 -O3/-O0 for mips64el.

And I am building LLVM 5.0.2 for X86, if still *not* able to reproduce, it might be reduced the issue to [6, +∞) version range. I will dig deep-into which commit/review bring in the issue.

Martin_Buchholz · September 11, 2018, 7:03pm

https://openjdk.markmail.org/thread/rwfcd6df6vhzli5m

Leslie_Zhai2 · September 12, 2018, 11:10am

Reported https://bugs.llvm.org/show_bug.cgi?id=38911

Thanks,

Leslie Zhai

Martin_Buchholz · September 12, 2018, 11:47am

I tend to think that clang's optimization was valid, and if placing
data into .rodata has any value at all (why else does .rodata exist?)
then the optimization is valuable.

Leslie_Zhai2 · September 12, 2018, 11:55am

Hi Martin,

Thanks for your response!

I tend to think that clang's optimization was valid, and if placing
data into .rodata has any value at all (why else does .rodata exist?)

placing data into .rodata is *not* wrong! But it needs to double check the setter of isReadOnly() if condition in the llvm/lib/Analysis/MemoryDependenceAnalysis.cpp and llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
Thanks,
Leslie Zhai

Martin_Buchholz · September 12, 2018, 12:12pm

Hi Leslie,

Hi Martin,

Thanks for your response!

I tend to think that clang's optimization was valid, and if placing
data into .rodata has any value at all (why else does .rodata exist?)

placing data into .rodata is *not* wrong! But it needs to double check the
setter of isReadOnly() if condition in the
llvm/lib/Analysis/MemoryDependenceAnalysis.cpp and
llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

I'm not a compiler engineer but .... The bug report should make it
clear why the new clang behavior is wrong. I want const globals to be
put into .rodata whenever possible - I WANT to get a segmentation
fault when placement new is used with the address of a const global.

akorobeynikov · September 12, 2018, 12:38pm

The code in question has an undefined behavior per C++ standard. And
the transformation is certainly valid.

Topic		Replies	Views
Compiling OpenJDK8 with LLVM for mips64el LLVM Dev List Archives	2	89	September 5, 2018
vmkit segmentation fault revision 107990 LLVM Dev List Archives	1	66	July 24, 2010
problems with the 2.6 branch under x86_64 LLVM Dev List Archives	0	64	August 31, 2009
[Bug 25921] New: 3.7 LLDB is broken on i686, while 3.8 (trunk) is just fine LLDB	1	57	January 11, 2016
Broken versions of GCC and other tools LLVM Dev List Archives	0	61	June 4, 2008

OpenJDK8 failed to work after compiled by LLVM 8 for X86

Related Topics