Crashes on Windows 8 with >4k stack frames

Hi All,

we have an issue with our LLVM-based JIT compiler - executing the compiled code corrupts memory (and subsequently crashes) if we alloca more than 4k of variables (more than 511 8-byte ints). The same code works on Windows 7 (32 and 64 bit), Linux, MacOS. We compile LLVM and our program with Microsoft's Visual Studio 2010. Both debug and release builds are affected.

The variables are created en-block at the beginning of the function with code looking like

for (i=0; i<513; ++i) {
   AllocaInst *variable =
        mBuilder.CreateAlloca(Type::getInt64Ty(mContext),0,"");
   mBuilder.CreateStore(GetConstI("INT4_8",0),variable);
}

We have not yet looked at the compiled machine code (same on Win 7 and 8, or differs?). But the 4k limit made us suspicious, as there were some bug reports - some still open - regarding this limit with LLVM [1,2].

So the question is - before digging into this for more days - is there some known issue with this, or does anyone have an idea what might go wrong?

Thanks,
Eph

[1] https://llvm.org/bugs/show_bug.cgi?id=2921
[2] https://llvm.org/bugs/show_bug.cgi?id=8919

Make sure the stack probes are emitted (__alloca / chkstk) and that
they indeed do probe the memory.

It's a known issue. I believe it's fixed in trunk however.
What LLVM version are you using?

     Nick

We tested on 3.4.2 and 3.5.1. Later versions are slightly problematic to test since they don't compile with VS2010. Do you happen to know if it's fixed in one of the released versions, or if there is a workaround (chkstk?) or a bug report online?

Thanks!
Eph

Thanks, chkstk was a very good pointer of what might go wrong! I assume LLVM optimizes the chain of alloca/store to a single alloca & memset. With stack growing down, memset working upwards, memset would touch a byte after the guarding page. Note sure if that assumption is correct, but it would explain what we see.

Not sure yet if chkstk is emitted, will try to find out.

Thanks,
Eph

With an old version of LLVM, this discussion and related patch may be relevant.

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20131021/192392.html

Hi,
Please see this bug report that I filed:
https://llvm.org/bugs/show_bug.cgi?id=23368

To quote myself:
"I think the fix went in after the 3.6 branch. I'll close this bug report and reopen if I hit it on trunk."

Thanks,
     Nick

Thanks Nick and all others, it's fixed now.

For the record, should anyone have a similar problem, the issue linked from yours [1] had a detailed analysis and provides patches that we were able to backport to 3.4.2.

- Eph

[1] https://llvm.org/bugs/show_bug.cgi?id=18582