This includes a need for potentially ABI breaking changes; when the next spectre vulnerability hits, we want users to get the mitigation automatically rather than having to learn that they need the mitigation at all.
The compatibility story would be that we’re ABI incompatible with GCC and with other versions of Clang with this flag.
I think this is going to scare people off using it. I was surprised to see this in here and we’re (as a distribution) definitely in the camp of “would use this mode if it exists”.
I was thinking that you can have multiple --config uses on the same command line, with the usual last-flag-wins behavior. Would that suffice for your needs?
It took us a lot of time and work from @mgorny et al to get config precedence and combinations working correctly. I’d be nervous about changing any of that. @mstorsjo’s reply covers a lot of this.
The issue with non-tunable omnibus flags is that users (and distros) turn off the flag (and therefore disable all child flags) if one of the child flags cause a package to not build.
I’m not sure if I follow this – if some constituent flag of -fhardened (or whatever) breaks a package, why wouldn’t you just do -fhardened -fno-constituent?
The option of having some default config file that is read for hardened options sounds good. If we want Clang to default to it on the system, we can then add that in to another config file. But I don’t know how it should then interact with this mechanism if a user wants to enable it cleanly when the system doesn’t.
I think this could work though? -fhardened (or whatever) has some default options, it reads from a system config file, and users can specify it in *FLAGS in their build system or via a wrapper or their own config file to enable it.
What we do right now (which is always enabled for some modes, and it gets more stuff added for our own hardened modes):
$ cat /etc/clang/gentoo-hardened.cfg
# Some of these options are added unconditionally, regardless of
# USE=hardened, for parity with sys-devel/gcc.
-Xarch_host -fstack-clash-protection
-Xarch_host -fstack-protector-strong
-fPIE
-include "/usr/include/gentoo/fortify.h"
# Options below are conditional on USE=hardened.
-Xarch_host -D_GLIBCXX_ASSERTIONS
# Analogue to GLIBCXX_ASSERTIONS
# https://libcxx.llvm.org/UsingLibcxx.html#assertions-mode
# https://libcxx.llvm.org/Hardening.html#using-hardened-mode
-Xarch_host -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE
$ cat /etc/clang/gentoo-hardened-ld.cfg
# Some of these options are added unconditionally, regardless of
# USE=hardened, for parity with sys-devel/gcc.
-Wl,-z,relro
-Wl,-z,now
# Options below are conditional on USE=hardened.
I prefer config style. In my experience, the C++ ecosystem is pretty complex. It is not practical to publish a one-for-all but not backward conforming solution and ask users to adopt it. Users won’t just silently refuse it. So I believe if we want to help more people instead of claiming we’re safe right now, it is better to make our solution configurable.
So this mode needs to set user expectations appropriately: your code breaking between compiler releases is a feature, not a bug.
In another direction, if this is what we decided, we’d better to move all the breaking changes to the hardened mode. In practice, we’ve already have some breaking changes. e.g., `enum-constexpr-conversion`, a lot of changes to make warnings on by default and other new checks. I mean, if possible, maybe we can move all such changes to hardened mode in the future. Then we’re able to get better backward compatibility. It is painful to fix these things.
And finally, I think ABI breaking change is a different level. We must be very careful for ABI breaking change. Recompile the world isn’t the thing the current C++ ecosystem can afford. Even if we want it for whatever reasons, we’d better discuss it separately in a different RFC. I don’t think we should mix this with safety proposals NOW.
Thanks a lot for the proposal! I think this addresses something really important in the C++ ecosystem right now and I am happy to see Clang positioning itself on that point.
I don’t want to chime into implementation specifics since I’m not really a compiler person and I trust Clang folks to know what’s best for Clang. Instead, I’d like to suggest three things that IMO should be design constraints of this proposal:
First, I think it’s really important for these hardening flags not to change ABI. Based on our experience with libc++, this is a core property that is required for the feature to be widely usable. For example, this means that you can adopt hardening on a per-translation unit basis, and you can enable it in parts of your codebase but not others. Not only is that useful from a convenience perspective, but there is also documented evidence showing that it’s necessary to deploy this at scale in a real-world code base (e.g. Google’s blog post about their adoption).
I think we should design this with the new “Hardened Implementation” notion introduced by C++26 (in p3471) in mind. Right now, that “hardened implementation” notion includes only standard library hardening, but I don’t think it will stop there. It’s also a really simple and useful model for users if they can just think of the flag as enabling the “standard” hardening feature. This entails that Clang’s flag should enable Standard library hardening.
We need to have a way to enable the different libc++ hardening modes (fast, extensive, debug), and we need this new Clang hardened flag to be compatible with users who select a custom mode. I don’t care strongly how that’s done, but it should be ergonomic. Our current way to enable hardening requires users to do e.g. -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE and that’s not great – it looks like a hack and we’d end up with a conflict if Clang’s hardening flag also enabled the fast libc++ hardening mode.
Like I said, I don’t want to suggest constraints on how this should be implemented, however I’d like to propose those three things as constraints on the design of this new Clang mode. This is based on our experience with libc++ hardening and other security efforts in the compiler in the past couple of years.
Thanks for the proposal. We’ve been thinking about something similar. Here are a few important things that came up:
The core problem that we wanted to solve was to give teams clear guidance on what compiler options they should use for security, and an easy way for auditors to verify that the settings are used. Most code generation settings (like trivial variable auto-initialization) are passed from under people’s feet and the auditors’ job is to verify they are not overridden by projects. Most warning flags are opt-in and the auditors’ job is to verify that they are enabled. An ideal implementation, for us, would be a single flag that enables everything that has to be enabled and diagnoses if parts of the configuration is overridden.
It is unlikely that open source projects will adopt -fhardened if it means something vastly different across platforms (especially if it means some platforms become pickier than others).
-fhardened should probably be versioned. There’s always an adoption cost to new security features, even when they’re as small as warnings. If the proposed implementation is config files, then we could consider multiple config files so that users could do, for instance, -fhardened=19.0.0, and upgrade when they have time. -fhardened should pick the latest and greatest.
This aligns well with discussions in the Memory Safety Working Group about streamlining security feature adoption.
One key challenge we should address: ABI compatibility determines if we can deploy a security feature in some of the environments we care about. Coupling the general security model with ABI breaks would limit its applicability in the very environments where security is important.
@AaronBallman There’s no documentation for what -mspeculative-load-hardening actually does
The name implies it is a global flag that inhibits load speculation? If those are the semantics it seems that making it an on by default part of a hypothetical -fhardened flag seems like it could potentially result in an extremely large perf impact.
I think we did consider that by default it should not enable abi-impacting options. However we think there is room for a “I want all of the safety at the cost of ABI” in some environment.
Yes. We might want to design that with contracts in mind because I don’t think we want two different mechanisms to control assertions.
There are also other STL (and libc, presumably) implementations to consider. I am not sure if it would be better to expose a compiler hardening level as a macro for library to use, or for the compiler to set something specific to each library.We did consider that individual flags should override the default set by -fhardened, but we did not consider the impact on -D.
We suspect finding a set of options that satisfies the cross product of all clang users, all libc++ modes, and all modes from other library implementation will be challenging so we certainly should consider having more granular ways to control libraries and/or labeled contracts.
From an end-user perspective, I think the fact that -fhardened could enable different features depending on the compiler used or even the version of the compiler, makes it difficult to use for a project.
As a distro maintainer, my opinion is that distros need to have explicit control over the flags used to build packages, so flags like this that can change significantly from release to release are not good to use.
I think it’s practically speaking dead code, but there is fairly extensive LLVM documentation on what this pass is and does. In short, it is a mitigation for information leaked through speculative side channels. Most folks who are concerned with hardening are more concerned about securing the front door, i.e. exploitable memory safety vulnerabilities, rather than protecting leaking secrets through the exhaust vent in this metaphor.
I think this is a really interesting perspective pointing to questions worth considering: Should the compiler have more high-level “intent-based” flags?
Let’s be honest, C compiler flags are not user-friendly. Compiler tend to expose low-level, specific, orthogonal flags, that are very far from the programmer’s high-level intent. If you are a distro maintainer, this can be useful, because the distro maintainer is in the business of setting the intention of the build and creating build flag policy. However, if you are a C/C++ programmer or compiler developer, you are forced to communicate through this low-level cflag protocol. Programmers are forced to copy-paste meaningless stanzas of common-sense debug info settings, optimization flags, and hardening settings.
There’s no high level way for the user to say “this is a production build, give me the current recommended best defaults”, and for compiler developers to ship new functionality without an expensive coordinated global flag migration across the entire C/C++ ecosystem (think about how we migrate to -std=c++N+1). Build systems attempt to fill the gap and define useful Release and Debug build presets, but these presets themselves become a form of tech debt, a leaky abstraction that must be worked around in order to enable the latest and greatest in LTO or split DWARF tech. It seems like there should be some room somewhere for us as compiler engineers in this ecosystem to express our opinions on best recommended settings and create a well-lit path for users.
-Og seems like obvious prior art for this kind of intent-based flag, but it’s an optimization setting, and those should only change behavior for programs with UB, which we permit.
To conclude, I think we should permit ourselves to have intent-based flags that change behavior over time, and if folks still think we shouldn’t, my question is, how else should we encode our evolving build configuration recommendations?
I don’t have a very strong opinion on this topic, just trying to give my perspective as a user. I think the high-level intent flags are always going to run into the same problems that we see with -Wall -Werror, where the differences between different compilers makes it very difficult for a project to support multiple compilers at once and use these options. Now, I think you can reasonably argue that you would be less likely to run into these issues with -fhardened since there will be less differences between compilers, but when there are issues I think it will be very difficult for inexperienced users (who I think are the target users for this flag) to figure out what is going on.
This is why if we were to add some sort of intent flags, I think they are best implemented as config files, because then at least users can look at the files and see what the flags are. The config files would also be something that is specific to clang and would let us avoid the whole situation of trying to figure out whether or not our -fhardened implementation needs to match gcc. The config files are also easier to version as someone suggested up thread, so you could more easily fix on a specific set of flags even if newer versions of clang wanted to add new hardening flags to its recommended list.
I am in favor of an -fhardened flag which tracks the latest recommended settings for hardened builds, even if this means that semantics changes between clang versions.
I am in the lucky situation that we have a single, blessed toolchain. We bootstrap our own clang toolchain, and every developer automatically downloads that toolchain as part of his build. We compile everything from source (including clang itself and libc++), so even an ABI-breaking flag would not be an issue for me. (Although, I never quite figured out how to use memory sanitizer, since I couldn’t figure out how to distribute an msan-annotated version of libc++ in our pre-compiled toolchain )
I am interested in adopting new hardening features. Currently, every time we upgrade to a new clang version, I spend quite a bit of time researching which new features might be interesting to us, how stable they are, etc. Having the best practices encoded in a -fhardened flag would save me that time. And in case a particular flag doesn’t quite work for us, I would just use -fno flag to opt-out of that particular hardening.
I think different users might have different goals with -fhardened.
Maybe we want to have different flavors of “hardened”?
To me the categorization by libc++ into “fast” (and most security critical), “extensive” and “debug” makes a lot of sense. Maybe we should have -fhardened=fast, -fhardened=extensive and -fhardened=debug? That would also implicitly solve @ldionne‘s topic on how to map -fhardened to the different libc++ modes.
-fhardened=fast would include the most security-critical but still reasonably fast optimizations (-ftrivial-auto-var-init, …).
-fhardened=extensive could include additional flags for security-critical apps for which runtime performance is a secondary concerns (e.g., flags to guard the “exhaust vent”, such as -mspeculative-load-hardening)
-fhardened=debug could go a step further and even include things like UB-sanitizer checks, similar to -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_DEBUG which checks “that the given argument satisfies the semantic requirements imposed by the C++ standard”, even if violating those requirements might not lead to a security issue per se.
Agree with all of @avogelsgesang’s points. And not version it or default to “the latest and greatest” as in @fay59 proposal for 2 reasons: (1) It’s easy to explore for people who are used to GCC’s -fhardened, because we’d reject the plain flag and note the modes in the error message. (2) It avoids expectations for compatibility.
In addition, I would like to point out that there are domain-specific hardening solutions provided by 3rd parties. They’d usually be distributed as compiler plugins, that inject/swap opt/codegen passes. I am not sure to which extend it needs consideration right now, but it might be valuable for them to detect the specified hardening mode through the plugin API.
We just talked about this in the memory safety workgroup meeting and I wanted to write here and expand on some of the things I said.
The first one is that users of -fhardened, by nature, are not C/C++ security experts, or else they would already just be using all the right options themselves. They are people who are motivated to have a secure code base, but that don’t necessarily have the time to learn all the things that Clang can do or make major changes. -fhardened ideally means “make my code as safe as possible given my constraints”. Our challenge is that “my constraints” is a very fuzzy one-size-fits-none measurement.
Here’s how I would put it: when we make a decision to enable a security feature in Clang, we have to be confident that it will be fine for 99.9% of code bases. For instance, while some vendors enable stack variable autoinit by default, we wouldn’t do it in upstream clang because we expect that there is some proportion of users for whom that would be a serious performance regression. IMO, for -fhardened, we should have a vibe of “it will be fine for 90% of people who try it”. That is, if we found all of the people I just talked about and we made them all turn on -fhardened in their projects, they would self-report that they were successful in an amount of time that feels proportional to the problem about 90% of the time. This isn’t a very clear line, but I think that it’s useful to identify some clear yeses and nos. For instance, I think it makes it a clear yes for stack protectors, and a clear no for ABI-breaking changes.
Reading all this, I’m wondering if we aren’t trying to run before learning how to walk.
There is a lot of knowledge about security and all the flags that are linked to it. Maybe the first step would be to already combine all this information on a documentation page which can easily be found.
At the same time, this will give us a list that we can quantify. For example: how many hits of this flag (and the one to disable it) can we find on GitHub, what’s the performance impact. (As an imperfect proxy for: how much impact would adding the flag cause?)
This could already help users without much security experience (including myself here) to take a first step while we figure out how to continue.
For sure, additional documentation is helpful and I would be happy to see someone take up the effort to document security related flags better and to have dedicated page to them. We will be gathering feedback for a while but that should not stop anyone from putting effort into more documentation.
I don’t think we should blocked on having other improved documentation. I am guessing if no such documentation arrived before the actual implementation is done then such documentation would be written as part of the implementation process.
)