Undefined Behavior Sanitizer (UBSan) is a useful compilation mode in Clang for finding uses of undefined behavior (e.g. signed integer overflow) and problematic C/C++ code (e.g. unsigned integer overflow). The default version of UBSan uses a compiler runtime that only works in userspace (e.g. it won’t work in the kernel or for embedded applications) and is not considered secure enough for use in production environments. To handle these other environments UBSan provides a trapping mode that emits trap instructions that immediately halts the application rather than calling into the UBSan runtime which normally diagnoses the problem and then carries on execution.
Unfortunately trapping UBSan has some deficiencies which make it hard to use. In particular:
Clang silently ignores the -fsanitize-trap=undefined flag when it’s passed without -fsanitize=undefined. This project would fix this as a “warm up task” to get familiar with the Clang codebase.
When a UBSan trap is hit with the debugger attached it is not convenient to figure out the reason UBSan trapped. For x86_64 and arm64 some information is encoded in the instruction but decoding this is very inconvenient. While LLDB could be taught to look at the instruction and decode the meaning this is brittle because it depends on undocumented compiler ABI. Instead we can build upon the __builtin_verbose_trap work to encode the reason for trapping inside the debug information.
Expected outcomes
When the -fsanitize-trap=undefined flag is passed on its own the compiler silently ignores it. Currently Clang requires that the -fsanitize-trap= flag is also passed. Clang should be taught to warn about this.
Teach Clang to emit the UBSan trap reasons in debug information on UBSan trap instructions similar to how __builtin_verbose_trap works.
Confirm LLDB is able to recognize the UBSan trap reasons and add tests for this.
Small (~10 hours) but can be extended if time allows
Project difficulty
Easy. This project would be good to a beginner to LLVM. If we end up having lots of extra time we can investigate improving the UBSan reasons by encoding other useful information that the compiler knows into the trap reason.
Do we really need ubsan version __builtin_verbose_trap? Can we just use it as is?
I think it might not be clear what I’m proposing. Trapping UBSan is already implemented in clang and being used in production. This proposal is about making debugging code built with this build mode of UBSan easier to work with.
This proposal uses the debug info emission implementation of __builtin_verbose_trap (which is also the approach used by -fbounds-safety).
I’m not proposing we add another builtin. The programmer doesn’t have to do anything different, this is a clang codegen change.
The current behavior of this seems intended and desirable. You can unconditionally specify -fsanitize-trap= if you want that mode, while allowing -fsanitize=undefined to be enabled or disabled for different files in a build. Maybe it needs better documentation?
I expect the items proposed to be fairly quick to implement (depending on mentee skill level). The rest of the work is potentially unbounded and much more open ended. That’s what the “10 hours” is trying (and kind of failing) to encapsulate. Do you foresee that being a problem?
Here are some concrete examples of what I think clang should warn about:
$ cat ubsan_overshift.c
int shift(int a, int b) {
return a >> b;
}
# There are no UBSan checks emitted.
$ clang -fsanitize-trap=undefined example.c -o - -S -emit-llvm
# There are no shift checks emitted.
$ clang -fsanitize=signed-integer-overflow -fsanitize-trap=shift ubsan_overshift.c
This seems like an unnecessary footgun. I think clang should warn when the checks specified in -fsanitize-trap= aren’t actually enabled.
The smallest GSOC projects are supposed to take 90 hours according to the FAQ. Proposals should probably try and stick to one of the standard project sizes.
Yes I think that could be a problem. As @boomanaiden154-1 mentions 90 hours is the minimum size for a small project. If you expect there to be significant effort to integrate, generate tests, etc, then those probably belong in your estimate.
I’d suggest clarifying in your post too, since students usually heavily tailor their proposals based on what included in it.
Another point I’ll raise is that something that is trivial for a long time contributor to do may be a challenge for a candidate. Often they are first year college students with little software development experience, and may not have even built LLVM before or ever contributed to a production code base.
I’d suggest being generous with your estimates, or even just vague with your estimates. The proposal the student/contributor submits will need to come up with those anyway and they may propose a project that devotes time to aspects you hadn’t really considered.
In my experience, the candidate’s own timetables usually account for their level of experience, and since we often discuss the proposals with the candidates we can usually advise them on more appropriate estimate if they’re out of whack.
That’s all just my $.02 though based on previous GSOC experience. Best of luck, as I’m pretty sure we have several embedded users that would appreciate improvements along these lines.
I’m interested in working on this project. I have about two years of experience with C and C++, along with hands-on experience using UBSan in systems programming projects. I believe this project would be a great introduction to getting familiar with the Clang codebase while also deepening my understanding of LLVM.
Given that the proposed project timeline is relatively short, I was wondering if I could start tackling the problem now and, along the way, propose additional improvements for GSoC 25. I’m open to suggestions on where to get started as well as drafting the GSoC proposal.
It would be annoying to have to know you need to also go disable -fsanitize-trap for each target that opted-out of sanitization.
And, why inconsistently, only for -fsanitize-trap, and not any other sanitizer-option which is applicable only if a particular sanitizer is actually enabled? Note that there’s other options that even take sanitizer names the same way, -fsanitize-merge-handlers=<sanitizer-kind>, -fsanitize-recover=<sanitizer-kind>, -fsanitize-skip-hot-cutoff=<sanitizer-kind>=<weight>.
Yet, if it was consistently an error to specify options for a disabled sanitizer, it would be even more annoying…
@ilovepi@boomanaiden154-1 Thanks for taking the time to read my proposal and provide feedback. This is the first time I’ve ever volunteered to be a GSoC mentor so I’m not surprised I got this a little wrong. I will update the estimated project size and also also include another high related task in the project proposal.
Unfortunately it seems discourse won’t let me edit my original post. @akorobeynikov Do you know if there’s a way to lift that restriction?
@jyknight Thanks for the example. It’s a lot clearly to me now what your concern is.
The proposal only focuses on -fsantize-trap because this proposal is focused around trapping UBSan. I’m not sure if -fsanitize-recover= is applicable to normal (userspace) UBSan because that continues without error by default so you don’t need different codegen (unlike ASan), although I take your point. We could make Clang emit a warning when -fsanitize-recover= is used on a sanitizer that isn’t enabled. The way I view this is we should start with the -fsanitize-trap= flag and extend later if we think it’s worth it.
Ah. I didn’t specify this in the proposal but what I’m proposing is the Clang driver would emit a warning, not an error. So by default the Clang driver will warn about something that is likely (but not always) a mistake, but it won’t break the build, and can be suppressed if the user doesn’t want to see the warning.
Does that seem reasonable to you? If you aren’t onboard with this general direction I may drop this task from this proposal because it’s not really a good warm-up task if its there’s going to be resistance from the community.
Given the size of the project I’m not super keen for work to start on it early because it risks me having to significantly rework the proposal.
OTOH, you can technically submit patches but these would be considered as part of normal open source contributions to LLVM and would not imply mentorship. Please note having accepted patches does not guarantee acceptance of your proposal but they might increase your chances.
Another question, I have never used diagnostics mechanism before, and I note that there are some DiagId in DiagnosticDriverKinds.inc. Should I define a new DiagId in it and use the new Id to replace llvm::errs?
The proposal has already been reworked to fit GSOC’s timelines, and is implementing features my own team has been considering. If you’re interested in this work, I’d encourage you to work w/ the mentor on making a good proposal, and collecting any additional details you think would help you write a better one.
As for disagreement w/ maintainers, the main disagreement I see involves existing flags and their interaction. How flags should interact is a part of the overall design, and something that can be handled during the course of the project (if not the proposal). We want them all to work well together, but I don’t think its a blocker for GSOC if the new functionality didn’t land w/ an existing flag name. If there are still questions about their interactions during the course of the project, an RFC is an appropriate way to address it with the rest of the community.
We should put up an RFC so the community can come to a consensus on what the correct behavior is. This is something I would guide the mentee to do and if it turns out the consensus is to keep the same behavior then I’d remove that part from the GSoC project.