Hi all,
Some background: I’m working on a project to replace a custom VM with various components of llvm. We have everything running just peachy keen with one recent exception, one of our executables crashes when attempting run a JIT’d function. We have llvm building and running on 64 bit Windows and Linux, using Visual Studio 2008 on Windows and gcc on Linux, and we have the llvm static libs linked into one of a DLLs, which is then linked to several different EXE’s. The DLL contains the code to compile to llvm IR, JIT and run code written in our proprietary language. Each EXE calls into this DLL the same way. The same chunk of IR, when JIT’d in 3 of the EXE’s runs perfectly, but in the last program, it dies in a call instruction out into an invalid memory location. All compiler and linker options are the same for all 4 exe’s. The one difference I’ve seen when debugging the assembly is that the 3 that work all have JIT function pointer addresses less than a 32 bit value but the one that is failing has a 64 bit address, as indicated in the snippet below:
000007FFFFC511D7 pop rbp
000007FFFFC511D8 ret
000007FFFFC511D9 sub rsp,20h
000007FFFFC511DD mov rcx,qword ptr [rbp-70h]
000007FFFFC511E1 mov edx,0FFFFFFFEh
000007FFFFC511E6 xor r8d,r8d
000007FFFFC511E9 call rsi
000007FFFFC511EB add rsp,20h
000007FFFFC511EF test al,1
000007FFFFC511F2 je 000007FFFFC511C3
000007FFFFC511F8 sub rsp,20h
000007FFFFC511FC mov rax,7FFFFC30030h
000007FFFFC51206 mov rcx,rdi
000007FFFFC51209 mov edx,0FFFFFFFEh
000007FFFFC5120E xor r8d,r8d
000007FFFFC51211 call rax
000007FFFFC51213 add rsp,20h
000007FFFFC51217 test al,1
000007FFFFC5121A je 000007FFFFC511C3
000007FFFFC51220 mov qword ptr [rbp-68h],rdi
000007FFFFC51224 mov eax,10h
000007FFFFC51229 call 0000080077B3F1D0
000007FFFFC5122E sub rsp,rax
000007FFFFC51231 mov rdx,rsp
000007FFFFC51234 mov qword ptr [rbp-0F0h],rdx
000007FFFFC5123B sub rsp,20h
The call instruction at 000007FFFFC51229 is the one that jumps into invalid memory at 80077B3F1D0. I’m not sure why this particular EXE causes llvm to use such large address values, but it looks like there might be some 32 bit vs 64 bit address calculation/offset problem when emitting the assembly.
The code that works looks like this:
0000000002931211 call rax
0000000002931213 add rsp,20h
0000000002931217 test al,1
000000000293121A je 00000000029311C3
0000000002931220 mov qword ptr [rbp-68h],rdi
0000000002931224 mov eax,10h
0000000002931229 call 0000000077B3F1D0
000000000293122E sub rsp,rax
0000000002931231 mov rdx,rsp
0000000002931234 mov qword ptr [rbp-0F0h],rdx
000000000293123B sub rsp,20h
000000000293123F mov r12,180071AD0h
0000000002931249 mov ecx,0FFFFFFEEh
000000000293124E xor r8d,r8d
0000000002931251 mov r9,29C02EAh
000000000293125B call r12
000000000293125E add rsp,20h
0000000002931262 mov eax,10h
0000000002931267 call 0000000077B3F1D0
000000000293126C sub rsp,rax
000000000293126F mov rdx,rsp
0000000002931272 mov qword ptr [rbp-58h],rdx
0000000002931276 sub rsp,20h
000000000293127A mov ecx,0FFFFFFEEh
000000000293127F xor r8d,r8d
0000000002931282 mov r9,29C02EAh
000000000293128C call r12
000000000293128F add rsp,20h
0000000002931293 mov eax,10h
0000000002931298 call 0000000077B3F1D0
000000000293129D sub rsp,rax
00000000029312A0 mov rax,rsp
And the code at 77B3F1D0 is this:
0000000077B3F1BE nop
0000000077B3F1BF nop
0000000077B3F1C0 int 3
0000000077B3F1C1 int 3
0000000077B3F1C2 int 3
0000000077B3F1C3 int 3
0000000077B3F1C4 int 3
0000000077B3F1C5 int 3
0000000077B3F1C6 nop word ptr [rax+rax]
0000000077B3F1D0 sub rsp,10h
0000000077B3F1D4 mov qword ptr [rsp],r10
0000000077B3F1D8 mov qword ptr [rsp+8],r11
0000000077B3F1DD xor r11,r11
0000000077B3F1E0 lea r10,[rsp+18h]
0000000077B3F1E5 sub r10,rax
0000000077B3F1E8 cmovb r10,r11
0000000077B3F1EC mov r11,qword ptr gs:[10h]
0000000077B3F1F5 cmp r10,r11
0000000077B3F1F8 jae 0000000077B3F210
0000000077B3F1FA and r10w,0F000h
0000000077B3F200 lea r11,[r11-1000h]
0000000077B3F207 mov byte ptr [r11],0
0000000077B3F20B cmp r10,r11
0000000077B3F20E jne 0000000077B3F200
0000000077B3F210 mov r10,qword ptr [rsp]
0000000077B3F214 mov r11,qword ptr [rsp+8]
0000000077B3F219 add rsp,10h
0000000077B3F21D ret
0000000077B3F21E nop
0000000077B3F21F nop
0000000077B3F220 int 3
0000000077B3F221 int 3
I searched the bug database for various topics but didn’t see anything specific other than one mention in bug 5201 to do with 32 bit address truncating. My dev system is a dual-core xeon with 16 gigs of ram. I’m no expert in how llvm works to output the asm, but I’m not afraid to delve into it to see what’s happening. Has anyone else run into this? Does anyone have a suggestion of where I might start to debug in the X86 emitter code? I’m not even sure how to create a test case that would use a large starting address for the JIT? Any help is greatly appreciated.
Thanks in advance,
.r.