[RFC] Add -fsanitize-address-disable-container-overflow flag to AddressSanitizer

Summary

This RFC proposes adding a new compiler flag -fsanitize-address-disable-container-overflow to AddressSanitizer that allows library developers to support disabling container overflow detection at compile time.

Motivation

AddressSanitizer’s container overflow detection can produce false positives in environments where all of the following are true:

  1. Application code is compiled with AddressSanitizer enabled
  2. Libraries/frameworks cannot be recompiled with sanitizers (e.g., system libraries, closed-source dependencies)
  3. Containers are accessed in sanitized and non-sanitized code

Currently, container overflow detection can be disabled at runtime via ASAN_OPTIONS=detect_container_overflow=0.

We’ve had feedback from users that it is not always possible to reliably control the runtime environment to set environment variables.

Previously it was possible to use a define on the command-line to control an #ifdef check used inside the libcxx container headers to disable the container overflow checks.

This was at best a hack as:

  • it used an internal-only __ prefixed preprocessor variable (which has now been removed from libcxx).
  • it was tied to a specific implementation of the containers and was not portable.

In order to make this a portable feature we need a compiler option that can be checked via __has_feature or some other mechanism at compile time to indicate that the end user wants to disable the container overflow check.

Proposed Solution

Compiler Flag

Add -fsanitize-address-disable-container-overflow flag that:

  • Only applies when -fsanitize=address or -fsanitize=kernel-address is enabled
  • Provides compile-time control over container overflow detection
  • Is marked as experimental since no standard library currently supports it

Feature Test Macro

Add __has_feature(sanitize_address_disable_container_overflow) that evaluates to:

  • true when the flag is enabled
  • false when the flag is disabled or is unimplemented

Recommended Usage Pattern

Library developers should use the pattern:

// In some container implementation
template<typename T>
void mycontainer<T>::resize(size_t new_size) {
    // ... actual resize logic
    // ... calculate beg, end, old_mid, new_mid

#if __has_feature(address_sanitizer) && 
    !__has_feature(sanitize_address_disable_container_overflow)

    __sanitizer_annotate_contiguous_container(beg, end, old_mid, new_mid);

#endif

    // ... any additional resize logic
}

This pattern ensures:

  • Annotations only included when AddressSanitizer is enabled
  • Backward compatibility with compilers that don’t support the new feature test
  • Users can disable container overflow checks when needed

User Compilation

Default: container overflow enabled

clang++ -fsanitize=address myapp.cpp

Disable container overflow to avoid false positives

clang++ -fsanitize=address -fsanitize-address-disable-container-overflow myapp.cpp

Backward Compatibility

The feature test pattern __has_feature(address_sanitizer) && !__has_feature(sanitize_address_disable_container_overflow) ensures:

  • Existing behaviour is preserved with or without a compiler that implements the flag
  • Libraries can adopt the pattern without breaking compatibility

Alternative Approaches Considered

  1. Runtime-only control: Current ASAN_OPTIONS=detect_container_overflow=0 approach has environment controllability limitations
  2. Providing a default implementation of const char* __asan_default_options() in a linkable object: This is complex to get correct and is very error prone.

Questions for Discussion

  1. Is this a good approach to support users who cannot recompile everything in their processes and cannot control the runtime environment?
  2. Should this be supported under -fsanitize=kernel-address?

Timeline

This feature is implemented and ready for review: [clang] Add flag to disable container overflow checks at compile time. by padriff · Pull Request #159662 · llvm/llvm-project · GitHub

Looking forward to community feedback on this approach to improving AddressSanitizer’s usability in heterogeneous environments.

1 Like

I understand that the flag is framed negatively to allow the feature test pattern to conveniently work in a backwards compatible manner:

#if __has_feature(address_sanitizer) && 
    !__has_feature(sanitize_address_disable_container_overflow)

but the downside is that -fsanitize-address-disable-container-overflow is inconsistent with other boolean sanitizer flags, which are typically named positively e.g.,
-fsanitize-memory-param-retval (disable with -fno-sanitize-memory-param-retval)
-fsanitize-undefined-trap-on-error (disable with -fno-sanitize-undefined-trap-on-error)

Would it be less confusing to separate the front-end flag name from the feature flag name, such that the front end name could be e.g., -fsanitize-address-container-overflow/-fno-sanitize-address-container-overflow?

Alternatively, -fsanitize-address-use-after-return= has {“never”, “runtime”, “always”}. If there was -fsanitize-address-container-overflow={"never","runtime"} that would 1) leave the door open for a future “always” mode 2) not come with any expectation that the feature name maps directly to the driver flag name.

Thanks for the feedback!

I tried a couple of different approaches to the flag, including something similar to the suggestion -fsanitize-address-container-overflow/-fno-sanitize-address-container-overflow. I could get all of them to work with the clang with the flag implemented.

Where I came unstuck was how to have the container code work as expected with a compiler that did not implement the flag. As soon as the test is for the presence of a flag it breaks down for a compiler that doesn’t implement it

#if __has_feature(address_sanitizer) && 
    __has_feature(sanitize_address_container_overflow)
   // unhappiness if the compiler doesn't support 
   // -fsanitize-address-container-overflow
#endif

Chatting to @ldionne and @varconst about this, it seemed this needed to work to be viable for them to adopt in in libcxx

I’m more than open to any ideas that would match the flag to a positive rather than a negative, but I’m scratching my head as to how to get it work that way.

The direct way to map a positive frontend flag to a negative feature flag is to have yet another feature, e.g., sanitize_address_configurable_container_overflow, that is always enabled with ASan.

The downside is the backwards-compatible feature test would then be a bit more complicated:

#if __has_feature(address_sanitizer) && 
    (!__has_feature(sanitize_address_configurable_container_overflow) ||
     __has_feature(sanitize_address_container_overflow))

There is somewhat of a precedent with this double-feature approach in [clang] [ubsan] add __has_feature for UBSan checks by fmayer · Pull Request #148310 · llvm/llvm-project · GitHub , which allows checks such as:

#if __has_feature(bool_sanitizer) || (!__has_feature(undefined_behavior_sanitizer_finegrained_feature_checks) && __has_feature(undefined_behavior_sanitizer))

Alternatively, my original thought (which might not be implementable) is: could we have

  • -fsanitize-address-container-overflow is a no-op
  • -fno-sanitize-address-container-overflow sets __has_feature(sanitize_address_disable_container_overflow)

Or likewise:

  • -fsanitize-address-container-overflow=runtime is a no-op
  • -fsanitize-address-container-overflow=never sets __has_feature(sanitize_address_disable_container_overflow)

In both cases, the backwards compatible feature test would remain the same as yours.

Generally, clang doesn’t want to add new has_feature for non standard C++ features. See Frontend: Define __SANITIZE_*__ macros for certain sanitizers. · llvm/llvm-project@568c23b · GitHub.

Also, I feel like your problem could be solved by coming up with a standardized define name, say __ASAN_NO_DETECT_OVERFLOW__, and does not really need a special frontend flag (you can just set it with -D). The __ would make sure that no one can use it for a different purpose. Maybe @MaskRay can comment on this as clang driver maintainer.

2 Likes

It’s between a library (e.g. libc++) and compiler-rt.

Compiler does not need to be involved at all.

libc++ can have an option here c++/v1/__config_site

My original request was for a standard #ifdef but I got push back. I’ll have to let @ldionne speak to why that was.

If the proposed option only applies to library code, like libc++, I agree that a macro would be better than a Clang driver option.

I think this is actually what we want. We’d just have to carry around a check for the compiler version until all versions of Clang supported by libc++ implement the check. Inside libc++, it would look like this:

#if __has_feature(address_sanitizer) && \
    (__has_feature(sanitize_address_container_overflow) || _LIBCPP_CLANG_VER < 2200)
   // -fsanitize-address-container-overflow
#endif

Hence, for older compilers, we’d simply assume that ASAN is controlled exclusively by -fsanitize=address (like we do today), and for recent compilers we’d have fine grained control of just the container sanitization feature.

Container ASAN checks are a general feature that can be used by user-defined containers too, they are not specific to libc++. Furthermore, the limitation of needing to have all of the process compiled with the checks enabled is a byproduct of the way these checks are implemented (with global state). Therefore, I don’t think it makes sense for libc++ to provide a switch that allows disabling just its own checks – that would be a hack.

Instead, providing a compiler flag and the corresponding __has_feature check lets any user-defined container also provide consistent behavior, and acknowledges that the need to sometimes disable the feature is not libc++ specific, but “compiler” specific (really compiler-rt specific). In principle, we could also use a macro like _COMPILER_RT_HAS_CONTAINER_OVERFLOW which would be equivalent to the __has_feature, but I don’t think we have precedent for this kind of compiler-rt <=> user library communication, and it feels a lot less polished than having a proper __has_feature flag.

TLDR, I quite like this proposal and my main comment would be to use the positive form for the __has_feature.

This isn’t a compiler feature though (and, as said above, it seems like even for non-standard compiler features, __has_feature is no longer recommended, CC @AaronBallman).

I also don’t understand what is “less polished” about a macro name. I don’t see any functional difference, when both the __has_feature and the macro name can be directly controlled by the user. Also, it isn’t really communication between compiler-rt and the user library, but between the compiler command line and the user library. That seems like what -D was designed for. (Even if it were, I don’t think there is a precedent for using __has_feature for compiler-rt<=>user library communication either.)

The two options appear to be:

Add a positive compiler flag

This allows some logic to tie the use of the flag to the existing sanitizer flags and for the check to be positive as detailed above.

However, this doesn’t appear to fit with how __has_feature or compiler flags should be used.


Move to a standard macro name

This would be defined by the user on the compiler command line using the standard -D syntax.

I don’t see a way that this could be a positive check as it would be entirely user supplied and independent of the compiler flag. The preprocessor logic in the source code should match the defines passed on the command-line otherwise it just gets confusing.

The check would end up being something like

#if __has_feature(address_sanitizer) && \
    ! _COMPILER_RT_DISABLE_CONTAINER_OVERFLOW

// Do sanitizer container overflow check code

#endif

So far comments have indicated that it is preferable for this to be a positive check rather than a negative one.

But, it has been indicated that a macro is more appropriate than a compiler switch.

I have a question on discoverability of the standard define: how do container library writers know that it exists and should / could be used.


Neither of these approaches ticks all the boxes, so are we able to agree on the approach?

I originally asked for a positive check, but I for one have been persuaded by the majority comments that a macro is better than a compiler switch for this use case. If having a negative check is the price of using macros, I am ok with it.

It sounds like the consensus is towards using a macro for disabling the checks; this sounds reasonable to me.

I think that using the positive form of the macro may be tricky. Could this rely on the user supplying -D__ASAN_ENABLE_CONTAINER_OVERFLOW__=0 and then using Builder.defineMacro("__ASAN_ENABLE_CONTAINER_OVERFLOW__"); to enable this by default (based on [this]) if the user has not explicitly set it already?

WRT discoverability, perhaps this could be achieved by adding notes to the flag description for detect_container_overflow [here], and to the container overflow hint [here].

It seems that the __sanitizer_*_contiguous_container functions are not in the public interface header currently - it looks like libcxx defines these itself where needed (for example [here]). Should these be added to the interface header so that other container implementations can get similar quality ASan diagnostics?