Guarantees on stack overflow

Hi list,

I'm wondering what guarantees I get if my program runs into a stack
overflow.

Background: I'm compiling security critical software via LLVM-IR. I can
guarantee (by program analysis, theorem proving, etc) that I have no
buffer overflows, and that the program is terminated in a controlled
way if it runs out of heap memory (calloc from libc has a clearly
defined semantics of returning NULL, or, in practice, the process may
be killed by the OS's OOM killer).

But what's about the stack? I cannot even estimate how much stack I
need, as this depends on LLVM optimization and backend passes.

From testing on a Linux x86-64 system, my experience is that I get a

SIGSEGV in these cases. But is this guaranteed? Or might it happen that
a stack overflow silently overwrites some memory and causes chaos and
security issues?

Thanks in advance for any help or links to documentation on this issue,
Peter

Hi Peter,

But what's about the stack? I cannot even estimate how much stack I
need, as this depends on LLVM optimization and backend passes.
From testing on a Linux x86-64 system, my experience is that I get a
SIGSEGV in these cases. But is this guaranteed? Or might it happen that
a stack overflow silently overwrites some memory and causes chaos and
security issues?

Generally the latter. Some targets (well, Windows) have stack probing,
which will make sure the function doesn't just jump a huge gap with
its stack allocation. Porting that to x86_64 Linux would probably be
very simple[*] (especially if you were willing to implement the probe
function yourself in your application), but I can't see any tests that
indicate someone already has.

Cheers.

Tim.

[*] It might even be as simple as defining the name of the function to
be called judging from tests on the existing stack-probes.ll testcase.

Hi list,

I'm wondering what guarantees I get if my program runs into a stack
overflow.

Background: I'm compiling security critical software via LLVM-IR. I can
guarantee (by program analysis, theorem proving, etc) that I have no
buffer overflows, and that the program is terminated in a controlled
way if it runs out of heap memory (calloc from libc has a clearly
defined semantics of returning NULL, or, in practice, the process may
be killed by the OS's OOM killer).

But what's about the stack? I cannot even estimate how much stack I
need, as this depends on LLVM optimization and backend passes.
From testing on a Linux x86-64 system, my experience is that I get a
SIGSEGV in these cases. But is this guaranteed? Or might it happen that
a stack overflow silently overwrites some memory and causes chaos and
security issues?

Hi Peter,

I do not know what, if anything, the documented policy of LLVM is. I
can tell you: in practice the default optimizations and code
generation will assume stack overflow doesn't happen and won't help
you hit the guard page(s), if there are any. For example, it's
possible for a function to adjust the stack pointer by a gigabyte and
call another function before touching any of the allocated stack
space. This typically leads to the callee's stack frame starting far
beyond any stack guard page and possibly overlapping other mapped
memory, so you can get memory corruption rather than a SIGSEGV.

There are some opt-in solutions to that. One is the "split-stack"
attribute (https://llvm.org/docs/SegmentedStacks.html), which
allocates more stack space from the heap if it runs out. Another
approach is the "probe-stack" attribute (sparsely documented
unfortunately, though it's in the LangRef) which ensures each page the
stack frame is touched once before the function code starts executing.
This ensures you hit a guard page if you have one, before clobbering
any other memory. You have to provide the function to actually do the
probing (and its ABI appears undocumented at a glance) but the call
will be inserted for every frame larger than page size.

Both of these need backend cooperation, I doubt all backends support
split stacks and as far as I know only the x86 backend supports stack
probes. It's also somewhat dependent on the the operating system. I
know the Rust compiler uses stack probing on x86(-64) Linux and used
segmented stacks there in the past, so for that target you should be
good.

Cheers,
Robin