[RFC] Clarifying Semantic Assumptions for Custom Allocators

Motivation

While working on formal verification with Alive2, we observed a mismatch between LLVM optimizations and the assumptions made by verification tools when reasoning about calls to custom allocator functions. In particular, the presence or absence of control-flow attributes such as willreturn changed whether a call was considered as potentially non-returning, which in turn affected both optimization results and verification outcomes.

This revealed a broader issue: LLVM IR relies on implicit semantic assumptions about allocator calls that are not clearly specified in LangRef. As a result, optimizations may rely on undocumented properties, verification tools must guess missing semantics, and users cannot know which guarantees their custom allocators must provide.

This RFC aims to clarify what semantic guarantees (if any) LLVM assumes for custom allocator calls, especially along the following three axes:

  1. Termination: Does the call return, or may it diverge?
  2. Exceptions: Can the call unwind or throw?
  3. Side Effects: Does the call have observable effects beyond memory allocation?

Definitions

In this RFC, the term custom allocator refers to any function call that LLVM treats as an allocator due to the presence of allocator-related attributes, such as:

  • allockind, allocptr, alloc-family, allocsize, or similar

This definition does not assume any specific control-flow or effect behavior beyond what is explicitly stated by attributes.


Scope

This RFC focuses on one question:

What semantic guarantees does LLVM assume for custom allocator calls, and where should these be specified?

It does not attempt to:

  • Redefine all allocator-related attributes
  • Specify malloc/free families
  • Redesign effect or control-flow attributes (e.g., willreturn, nounwind)

Problem Statement

In verification experiments, a custom allocator function without control-flow attributes such as willreturn was conservatively modeled as potentially non-returning.

As a result, transformations that removed or simplified such calls failed to verify, because the call was considered unable to return.

This behavior was observed in Alive2 (see PR #1252). When willreturn was added to the custom allocator declaration, the verification succeeded.

This concrete case shows that the behavior of allocator calls depends critically on whether control-flow attributes are present, and that different tools may make different assumptions in their absence.

define i64 @src() {
#0:
  %stack = call ptr @myalloc() allockind(alloc)
  %sz = objectsize ptr %stack, i1 0, i1 0
  ret i64 %sz
}
=>
define i64 @tgt() {
#0:
  ret i64 -1
}

Transformation doesn't verify!
ERROR: Source and target don't have the same return domain

Example:

Source:
  ptr %stack = function did not return!

Otherwise, when we used a custom allocator with a control-flow attribute such as willreturn, the verification succeeded.

declare ptr @myalloc() willreturn allockind(alloc)
define i64 @src() {
#0:
  %stack = call ptr @myalloc() willreturn allockind(alloc)
  %sz = objectsize ptr %stack, i1 0, i1 0
  ret i64 %sz
}
=>
define i64 @tgt() {
#0:
  ret i64 -1
}

Transformation seems to be correct!

Proposed Specification Options

This section proposes two possible directions for LangRef clarification.

Option A: Conservative Semantics

Allocator-related attributes do not imply any control-flow or effect guarantees.

  • Termination: May diverge unless explicitly marked willreturn.
  • Exceptions: May unwind unless explicitly marked nounwind.
  • Side Effects: May have side effects beyond allocation unless restricted by effect attributes.

Option B: Implicit Allocator Guarantees

Calls treated as allocators are assumed to satisfy certain default guarantees, such as:

  • Termination: Assumed to return unless explicitly marked otherwise.
  • Exceptions: Assumed not to unwind unless explicitly marked.
  • Side Effects: Assumed to have no observable effects beyond allocation and possible failure.

Questions for the Community

  1. Should allocator-related attributes imply any termination guarantees by default?
  2. Should they imply absence of unwinding or side effects beyond allocation?
  3. Are current optimizations relying on undocumented assumptions?

CC : @nlopes – Thanks for your time and feedback!

My understanding of allocator semantics in LLVM is that we make no a-priori assumptions about side-effects, unwinding, etc of allocators. However, allocations can be elided independently of any side-effects they may have. Allowing this is really the core purpose of these attributes, as allocators commonly can have side effects and/or can unwind.

I’m sure @RalfJung has some thoughts on how exactly allocator elision can be formalized, but I guess the model is something like a non-deterministic choice before each allocator call, such that either the allocator is called, or else the allocation is served from LLVM’s own magic memory allocator.

See also [expr.new] for the C++ spec wording that allows this for that particular language. Other languages that want to use allocator elision need to have similar provisions (e.g. see GlobalAlloc in std::alloc - Rust for Rust).

1 Like

It’s a question about the semantics of exposed function __attribute__ in C. The documentation doesn’t explicit say that the compiler is free to assume that allocation functions must terminate (Common Function Attributes (Using the GNU Compiler Collection (GCC))).

In clang we added -fno-assume-sane-operator-new a while ago, but I think it never got documented.

I’m ok with allocation attributes implying willreturn, but we have to document that.

1 Like

Which attributes precisely are you referring to here? If it’s __attribute__((malloc)), then I believe that corresponds to return-position noalias in LLVM, which in principle only carries aliasing guarantees. Allocator elision is bound to the allockind and "alloc-family" attributes.

Just to be explicit on this point, I don’t think that the allocator attributes imply any of inaccessiblememonly, nounwind or willreturn. Allocator elision is an independent mechanism that does not require or impose side-effect freedom.

LLVM does infer all of inaccessiblememonly, nounwind and willreturn on malloc() in C, but this an inference about specific, named libc functions, and not related to generic allocator handling.

1 Like

ok, then your suggestion is to model function calls with this attribute to have non-deterministic behavior w.r.t. to termination and exceptions, which would justify elision.

That approach works. But we should document that in LangRef so frontend writers are aware of it (though I hope in practice it won’t matter).

1 Like

Thank you for the helpful feedback.

I understand that each attribute is independent, and that allocator elision is a separate mechanism.

I agree with modeling allocator calls nondeterministically. However, I could not find any place in LangRef that describes this nondeterministic semantic model.

If this is the intended semantics, it should be written down explicitly in LangRef.

Documenting this would make the behavior clear to frontend authors, tool developers, and verification tools, rather than leaving it implicit.

Yeah, that’s the key part. There’s a lot more discussion in this Rust issue.

One question we did not resolve there is whether the compiler is allowed to “make up” new allocation requests that aren’t in the source. It also wasn’t clear whether the compiler is allowed to merge an allocate and a subsequent realloc, turning them into a single allocate with the final size.

Indeed, the LangRef is quite incomplete. Someone will have to add it there. :slight_smile:

1 Like

I’ve put up [LangRef] Mention allocation elision by nikic · Pull Request #177592 · llvm/llvm-project · GitHub to document allocation elision in LangRef.

2 Likes