RFC: Dynamic Debugging for C++: Step through unoptimized code in optimized builds

LLVM Project
, ,
1

Overview

We’d like to gather feedback on implementing an equivalent of MSVC’s new Dynamic Debugging feature in LLVM. Dynamic Debugging lets developers step through unoptimized code from optimized binaries. Many of our users are interested in this feature as it largely removes the historical trade-off of runtime performance for improved debuggability.

The full unoptimized binary is compiled ahead of time along with the optimized version, and all of its globals reference the optimized binary (both data and code - so unoptimized functions call optimized ones). While debugging, when a breakpoint is requested the debugger patches the optimized function entry to jump to the unoptimized version. Because unoptimized functions call optimized versions, continuing execution returns execution to the optimized binary.

We’ve built a working proof of concept. This RFC is to gather opinions on our design for LLVM and our current implementation, which requires coordination from compiler, linker, and debugger.

Nested ELF design

We are currently only looking at ELF support. The broad idea is that the unoptimized program (“inner ELF”) is compiled and stored in a new .debug_llvm_dyndbg section in the optimized version (“outer ELF”). This differs from MSVC’s approach which generates two binaries. “Nesting” the ELFs allows build systems and other tools to continue to manage a single object file per translation unit.

High level nested ELF design:

  • In the outer (optimized) module, globals with internal linkage not in a COMDAT are “promoted” so they can be referenced by the inner module. We implement this by introducing aliases with external linkage and suffixes unique to the translation unit. The suffix is currently “.dyndbg.<hash of directory, file, and driver flags>” (we’re open to other spellings).

  • The inner (unoptimized) module contains unoptimized copies of the outer module’s functions. Their names are prefixed with “__dyndbg.” (again, open to other spellings).

  • Declarations corresponding to the originally external and now-promoted globals in the outer module are added to the inner module. All global references (function and data) in the inner module refer to the outer module.

  • After linking, the inner ELF is still essentially an ET_REL object. It’s the debugger’s job to extract, load, and apply relocations for the inner ELF.

This diagram illustrates the result of compiling a simple source example:

image
image1284×702 45 KB

Compiler considerations

In our current implementation Clang clones the input module at after CodeGen (clang::emitBackendOutput) without global variable definitions, renames the functions as appropriate, and generates aliases as needed in the original (to-be-optimized) module. A seperate optimization pipeline (emitAssembly), set to O0, is run on the unoptimized module. The binary output of which is embedded into the to-be-optimized module (using embedBufferInModule) to be embedded in a .debug_llvm_dyndbg section.

Handling the cloned module here in the front-end simplifies some aspects (rather than teaching the LLVM backend code how to handle two modules), but has its own drawbacks. Most importantly, this approach makes it very difficult to share sections we otherwise might, such as debug strings or debug type units. I’m not certain how LTO would be supported this way either, though thus far LTO support has been a non-goal. Changes there may affect the implementation should not impact the design of it (the nested ELF approach).

Beyond the front end there are still some changes needed to LLVM itself. We can’t have any optimisations that change function interfaces at all, and any function cloning (e.g., function specialisation creating an local version of a global function) needs to be disabled as those functions won’t have corresponding unoptimized versions (and in the case of function specialisation it wouldn’t be safe to call the unoptimized global version after setting up a call to the optimized version, as the local version may have had its ABI messed with). Having external aliases for all the functions prevents the former but not the latter. GlobalOpt will replace some aliases with the aliasee, which is undesirable, and also needs to be explicitly disabled. Both of those behaviours could be controlled with a new function attribute, however, I am not proposing we re-open the “noipa” attribute discussion at this point. Given that external aliases provided a barrier to most interprocedural optimisations, and (as far as I’m aware) it’s just those two optimisations we need to control, I propose we go for something more targeted (that could be superceded by a noipa attribute in future).

On x86_64 we need functions to be at least 5 bytes large so our debugger can patch the entry to optimized functions to the unoptimized versions as needed. We propose adding another stringy function attribute to control this (tail-pad-to-size=<min byte size>).

We think it would be useful to add version information, for example in a .note.

Overall, Clang needs a small and localised change, and very few changes to LLVM itself are needed with the current design.

Linker considerations

The current implementation in ELF LLD basically performs a relocatable link of the unoptimized inner modules within the regular link of the optimized outer modules. The result of this inner relocatable link is placed in the .debug_llvm_dyndbg output section of the optimized outer link. The inner relocatable link uses the option --force-group-allocation to resolve groups and enables merging of the input sections. This helps to reduce the size of the output especially for inputs that use -ffunction-sections.

The key changes required in ELF LLD are to support the nested linking and to handle the symbol dependencies from the unoptimized inner modules to the optimized outer modules and the final executable output.

Debugger considerations

We want to ensure the design leaves room for implementation in LLDB, so we would greatly value feedback of LLDB/Apple folks on this.

From our debugger folks - broadly, a debugger that supports dynamic debugging must:

  • Read the relocations from the inner (unoptimised) ELF and apply them.

  • Load the .text of the inner (unoptimised) ELF into the debuggee’s memory.

  • Using DW_TAG_inlined_subroutine, build a map from each inlined function to a set of parent functions that inline it.

When setting a file/line breakpoint for a non-inlined function the Debugger should:

  • Lookup the address in the inner ELF’s line information and set the breakpoint in the unoptimised code.

  • Lookup the address in the outer ELF’s line information and patch the optimised function with a detour to the unoptimised function.

When setting a file/line breakpoint for an inlined function the Debugger should:

  • Find the possible parent functions for the inlined function using the inliners map.

  • Repeat the process for non-inlined functions for every possible parent function.

Function breakpoints can use similar mechanisms as file/line breakpoints, where the address lookup is done using symbol information instead. Address breakpoints do not need any special handling.

Users may create breakpoints that require patching of the same function. Due to this, patches in optimised functions should be reference counted. Patches can be removed when the reference count drops to zero.

Function calls in the unoptimised code will call functions in the optimised code. The Debugger must force execution back to the unoptimised code when stepping into function calls from unoptimized code. The Debugger should:

  • Put a temporary breakpoint on the call target.

  • After stopping at the call target, map the PC to the corresponding function in the unoptimised code.

  • Set the PC to the start of the function in the unoptimised code.

The exact timing of the detour patching needs to considered. If the PC of any threads is currently in a function to be patched, it may not safe to apply. A fallback mechanism that uses temporary breakpoints and sets the PC in the Debugger when the breakpoint is hit can be used until the function is safe to patch. While using the fallback mechanism, the Debugger should also put a breakpoint in the optimised code. This is for the case where a thread may already be executing the function and will never end up in the unoptimised code.

There are some considerations around attaching and detaching. When detaching, the Debugger should remove any patches it has added to optimised code. Debuggers can decide whether to remove the unoptimised code from the debuggee’s memory when detaching. A Debugger could, for example, leave the code and some metadata behind so that if a user re-attaches later the initial loading and relocating doesn’t have to be done again.

Resource usage

The costs of compiling with dynamic debugging are not straightfowardly “unoptimized plus optimized build”. In terms of size, the inner ELF remains relocatable after the outer ELF is linked, so it contains many relocations. The outer ELF contains many more functions than it would under normal optimizations (we can’t delete the optimized definitions as they’re called from the inner ELF), and those come with extra debug info.

  • Adding -fdynamic-debugging to a -O3 -g build results in an executable file size increase of +189.58% (and the increase for object files is more pronounced), geomean over CTMark projects.

Compile time can be fairly significantly impacted depending on the source. C++ parsing only happens once, and the codegen/emission time of the inner ELF is comparible to a normal unoptimzed build. However, the outer module can take significantly longer to compile than a normal optimized build due to the (sometimes vast) number of function definitions that are preserved that would otherwise be fully optimized away. Optimizations, ISel, and object emission all take longer as a result.

  • Adding -fdynamic-debugging to a -O3 -g build results in an compile time increase of +14.81%, geomean over CTMark projects (measured in instructions:u sampled, as on compile-time-tracker). The range across the CTMark codebases is from less than +2% to nearly +40% increase, indicating this is highly dependant on the codebase.

Linking of output with dynamic debugging will have a significant memory usage overhead.

Summary

In summary, this feature would allow developers to see both the benefits of running optimized builds and debugging unoptimized builds in one ELF. It requires coordination from the compiler, linker, and debugger, so we are seeking feedback from contributors in all these areas.

Thanks for reading,
- Orlando (@OCHyams) and Andrew (@nga888)

7 Likes
Visual Studio Dynamic Debugging
2

This would be a very useful feature to have, so thank you for pursuing this!

One mild concern I have is that users might expect unoptimized version to be actual debug version, with different value for NDEBUG and friends, but this would be a serious ODR problem. I looked at MSVC documentation, but it doesn’t seem to say anything on the matter. Can you confirm what MSVC behavior is? Do you think there are design options w.r.t. NDEBUG and similar macros, or it’s a clear cut?

1 Like
3

re: noipa… I was rathing hoping this’d be an excuse for you folks to pick up that work (I don’t think it’s /that/ bad/much work - I just ran out of steam). But I see you want the optimized code to allow itself to be inlined, but not to change its signature/contract - so that’s not “noipa” (inlining is the canonical IPA) so it doesn’t really fit your needs unless we did separate out inlining from ipa (we do have a noinline attribute, so we could argue that they are distinct behaviors and have noipa have no impact on full inlining (hopefully still classify partial inlining as “ipa”)). Would be a bit unfortunate - it’d be nice if no-inlining fell out of noipa for other reasons - though makes it less relevant to this situation.

Re: the whole build model. Sounds pretty complex, infrastructure-wise.

I assume there’s good reasons this doesn’t “just” rename the functions and keep them in the same module? (virtual address space? the desire to apply relocations lazily? some other things?) be good to document why that’s not a good choice/how much that costs/how much we gain by not doing that that justifies the complexity of embedding the other binary in this unlinked form.

1 Like
4

This is really exciting work, thanks for putting together such a thorough RFC. I really like the nested ELF approach over the two binary design, keeping one `.o` per TU means, build systems, caching, etc, all just work without changes. And core property that unoptimized functions call out to optimized ones is what makes this practical over shiping both builds side by side.

A couple of things I am interested in from debug info perspective, since we are keeping function definitions alive in the outer ELF that the optimizer would normally delete (because the inner ELF calls into them), does that bloat the outer’s DWARF noticeably beyond the +190% binary size you already measured? I am wondering if there’s room to emit thinner debug info for those kept-alive-only-as-call-targets functions. And also about the Inlining story in practice, for heavily templated C++ where a function gets inlined into dozens of parents, does the debugger patch all the parents eagerly when you set a breakpoint on the inlinee, or should it do something lazier?

But overall this sounds exciting to me! Thanks!

1 Like
5

Somebody brought this RFC to my attention, and I don’t think this is going into the right direction. I’m not particularly fond of how MSVC implemented their /dynamicdepot feature, especially in terms of compile times and vast increase of binary output size.

In my opinion, /dynamicdeopt has a very large up-front tax, even though you’re not going to use it on >95% of your code during a typical debugging session (think AAA projects the scale of UE5 or Fortnite). It’s essentially wasting a ton of resources on things you aren’t going to need anyway.

Therefore, I don’t think “copying” that approach, even though it’s done differently internally, is a particularly strong idea. I’d rather like to see things done fully dynamically, only in places where you’re going to need it, e.g. when setting a breakpoint somewhere.

Live++ and its hot-deoptimize functionality can already do that, on a per-translation unit basis (or whole module if you want to), so it’s definitely doable.

Lastly, similar to how the situation is with /dynamicdeopt, this won’t play nice with hot-reload tools such as Live++, since both dynamic debugging and hot-reload will fight over who gets to patch a function, and it can’t be both at the same time. It would be lovely if this could be taken into account.

Full disclaimer: I’m the developer of Live++.

1 Like
6

Thanks @Endill that’s a good point. I don’t think we’ve got much wiggle room there because function signatures may be be different if you set different macro/definitions. And if we could work around that somehow, supporting that would mean having to parse C++ twice, which we currently don’t need to do.

Can you confirm what MSVC behavior is?

The MS docs say:

The compiler flags that are used for the deoptimized version are the same as the flags that are used for the optimized version, except for optimization flags and /dynamicdeopt. This behavior means that any flags that you set to define macros, and so on, are set in the deoptimized version too.

1 Like
7

Hi @dblaikie,

The key concern for us is virtual address space, i.e. we want to be able to do partial patching with the unoptimized code. Therefore, at a minimum the unoptimized versions of the code need to be kept separate. We did consider using separate distinctly named sections but it seemed “cleaner” to contain the unoptimized parts in its own embedded ELF. This also means that the ELF output looks more “normal” apart from this additional dynamic debugging section. We decided to take this approach in both the compiler and the linker. Thus far, this embedded ELF approach has not been too complex in our prototype.

2 Likes
8

Yeah that’s correct that we want inlining in the optimized code. I am a bit worried that getting consensus on the scope of noipa and how it interacts with inlining could be difficult, and as you’ve pointed out I’m not certain all definitions of noipa would be useful for this use-case.

That’s not to say we shouldn’t try if you think there’s a strong case here, but my gut feeling is it’s orthogonal, or at least that other use-cases may require it to be.

(and thanks all for the replies, I’ll get to the rest soon!)

1 Like
9

Just a tiny comment: Nearly doubling the binary size (you said +189%) will make this a non-starter for many of our larger projects, which are struggling with a 1x binary size.

2 Likes
10

Hi @phyBrackets,

does that bloat the outer’s DWARF noticeably beyond the +190% binary size you already measured?

The +190% includes the DWARF, but generally yes the additional function definitions do increase the size of the DWARF. Here’s the debug sections that bloaty highlights (this is just an excerpt) on a large codebase comparing the outer ELF to a normal optimized ELF -

    FILE SIZE        VM SIZE
 --------------  --------------
   +50%  +321Mi  [ = ]       0    .debug_info
   +72%  +129Mi  [ = ]       0    .debug_loclists
   +94%  +126Mi  [ = ]       0    .debug_line
   +86% +50.8Mi  [ = ]       0    .debug_addr
  +198% +35.9Mi  [ = ]       0    .debug_aranges
   +59% +31.3Mi  [ = ]       0    .debug_rnglists
   +19% +28.9Mi  [ = ]       0    .debug_str
  +8.7% +2.48Mi  [ = ]       0    .debug_str_offsets
  +3.3%  +182Ki  [ = ]       0    .debug_abbrev

I am wondering if there’s room to emit thinner debug info for those kept-alive-only-as-call-targets functions.

That’s an interesting idea, I think something like that could be possible, though it would impact user experience if there’s a crash in one of those functions. There is definitley scope for exploring options that reduce the nested ELF size and any that impose a trade-off could be could be made optional / opt-in.

Looking at their docs, MSVC has an undocumented flag /d2DDTrimInlines which “reduces the iteration time overhead […]. The switch also removes the ability to step into small __forceinline/inline functions.” I’m not certain what the exact behaviour of the flag is, but it perhaps suggests they’ve run into the same sort of questions.

And also about the Inlining story in practice, for heavily templated C++ where a function gets inlined into dozens of parents, does the debugger patch all the parents eagerly when you set a breakpoint on the inlinee, or should it do something lazier?

Yes it patches all the parents. This problem isn’t unique to dynamic debugging though, as setting source breakpoints in inlined functions requires setting breakpoints at all inline sites, and similarly setting source breakpoints in templated code requires breakpoints in each instantiation.

It’s not clear that the patching is an issue or what the trade offs would be of a lazier approach. As long as it’s not a blocker/requirement for upstream we’re happy to explore that later if it’s found to be an issue.

2 Likes
11

Thanks @OCHyams for the detailed breakdown Orlando,. The +321 MB on .debug_info alone is eye opening imo but makes sense given how many definitions survive that normally wouldn’t. Good to know MSVC is hitting similar questions with /d2DDTrimInlines, an opt-in knob for size vs debuggability sounds like the right shape for that.

2 Likes
12

Hi @MolecularMatters, thanks for joining in the discussion.

We’ve had requests to support Dynamic Debugging from game studios with large game engines, including UE users, who have tried the MS feature. They feel the trade-offs are acceptable and we’re unsure whether the trade-offs of an alternative approach would be acceptable. We think that another benefit of the approach we’ve proposed is that the trade-offs are clear to understand.

In terms of tools fighting over patching, yes it could be an issue. I think the tools would probably need to coordinate to make that work.

1 Like
13

Hi @cmtice,

Are you able to go into more detail in what way the size is an issue?

1 Like
14

Oh, and since I’d be remiss if it wasn’t mentioned:

Have you considered JITing instead of building the whole program unoptimized up-front? (I imagine you have and it’s a bit late given how much you’ve already invested in this direction I doubt it’s likely to be dropped to try something new from scratch)

Something like a debugger that could read compile_commands.json, the program built with the “preserve optimized, out of line definitions” thing already described here, but then the unoptimized code is only built on-demand somehow (ORC JIT has a lot of layers built for maximum laziness/caching, integration with the CAS at some point if it doesn’t have it already - wonder if Apple can talk about whether they’re investing in this area for potential collaboration).

1 Like
15

Hello @Orlando275 :waving_hand:

I don’t want to misinterpret, but what your users are probably asking for is improved debugging experience for optimized code, not implementing MSVC’s Dynamic Debugging. You don’t have to copy their design.

I am simply wondering if we couldn’t come up with something less involved and with a smaller tax upfront, as compared to what your RFC proposes. I think the part about keeping around optimized symbols and avoiding certain optimizations is good to have in LLVM, to allow for later runtime patching. However implementing Microsoft’s “brute-force” approach seems way overkill in my view. I am not sure users who ask for this really understand what they are signing for.

Would it be possible to study / offer alternatatives? Something more “dynamic” like the flag says, where some work is done partially upfront, and some other work is done at debug time, on demand? I feel that would be a more pragmatic solution rather than a brute-force “compute-all-possible-outcomes-upfront” kind of solution.

There could be several approaches I think to achieve this:

  • Keeping the part about optimization constraints is good in your RFC.
  • Can we store LLVM-IR instead of fully codegen’ing the module? That would only induce a small tax upfront for build times (serializing the IR), and a bigger tax for storage. At debug time you can codegen the module on-demand, by invoking the compiler again from the debugger using the information stored by -frecord-gcc-switches. Most of the time you’d want to debug a local build, so the compiler would also be available on the machine where the debugging happens. I can’t imagine many cases where a user debugs a build but does not have the compiler locally.
  • As an alternate “dynamic” approach, you can simply invoke the -frecord-gcc-switches compiler command-line (by extracting it from the .o file) and fiddle a bit with the flags to build a -O0 temporary .o. Same as the above this would generate a small loadable module than can then be injected into the running process (what @dblaikie says)
  • Or just provide a perpetual Live++ licence for PS5 to your users who ask for it, like Epic Games does in Unreal Engine. What you are proposing here already works today in Live++, but with on-demand compilation, and on most game platforms. If only a handful of users want this, this will be probably several orders of magnitude cheaper than landing the RFC (both in terms of time and money) and then dealing with all the maintenance afterwards.
  • Also this problem of debugging optimized code was one of the reasons my RFC around the long-term vision on build times aims to solve. Your current RFC exists because nobody really wants to invest into the build pipeline in the first place (change the build model). A clangd-style compiler daemon with a DAP adapter could provide unoptimized functions on the fly, when needed, iff the build pipeline was structured around that.
1 Like
16

The simplest solution - which already works for several years - is to store the command-line in the .o and re-invoke the compiler by stripping optimization flags and adding -O0. The generated module can be injected in the process and currently running functions patched afterwards (which this RFC aims to do anyway). Building solely the TU also solves the issue of putting a breakpoint in leaf or inlined functions in core classes, which otherwise might require in the worst case deoptimizing large parts of the running process, to make sure the function is reachable.

1 Like
17

Purely out of curiosity (as this hasn’t been upstreamed), I wonder how does this compare to the prior approach:

Debugopt: Debugging fully optimized natively compiled programs using multistage instrumentation

Abstract:

“The accuracy of debugging information is crucial for source level debugging. However the debugging information may be inaccurate after sophisticated optimizations if the target program is compiled into native code. Hence, the efficiency of diagnosing software is affected due to inaccurate debugging information.
To address the issue, we propose Debugopt, a framework for debugging fully optimized natively compiled programs using multistage instrumentation. At compile time, Debugopt generates unoptimized programs with accurate debugging information and optimized programs. At debugging time, Debugopt dynamically replaces the execution of optimized programs with unoptimized programs. Debugopt is implemented on multiple architectures, including x86-32, x86-64, armv7 and mips3. Debugopt’s overhead is small during normal execution on a large range of benchmarks.”

FWIW, seems that some of the experience is similar:

Section 6.3. Code size

“The code size increase in the integration mode is due to the inclusion of unoptimized code, the debugging information of unoptimized code, symbol names and symbol tables of those unoptimized code. The sizes of these components vary across programs. For example, on x86-64, the increase by debugging information ranges from 27% in gobmk to 438% in dealII. The code size increase of the C++ programs is usually bigger than the code size increase of the C programs, because a template usually generates multiple copies of debugging information when it is instantiated with multiple parameters. For example, dealII is a C++ program which uses template classes heavily.”

For background:

“Debugopt supports two modes: integration mode and separation mode. These two modes are inspired by GDB [9] and Microsoft’s Visual Studio [10]: they separate the debugging information into special debugging information files. In the integration mode, both optimized object code and debugging meta code are linked into a single executable file or shared library. In the separation mode, optimized object code is linked into an executable file or a shared library, while debugging meta code is linked into a debugging library. The debugging library is a shared object which is dynamically loaded at debugging time. The separation mode is for debugging released programs whose debugging information is stripped after release.”

1 Like
18

so question becomes which IR you store. If it’s post optimization IR you’re in decompilation territory, you’d need to somehow reconstruct debuggable code from IR where inlining has happened, loops have been vectorized, and variables have been promoted to SSA values that no longer correspond to source level entities, the Source - IR mapping becomes fundamentally non trivial after passes like vectorization turn one source variable into multiple vector lanes across interleaved iterations. And if it’s a pre optimization IR, you’re storing something comparable in size to what the RFC proposes, just deferring the backend work to debug time, and now you need the compiler present on the debug machine, which isn’t always the case (remote debugging, crash dumps, embedded targets?).

I feel like it is clever approach, but also comes with subtle correctness issue, the source code at debug time must exactly match the source code at build time, byte for byte, ot the generated code won’t correspond to the binary you’re debugging, and I think with nested ELF approach, it isn’t the case.

1 Like
19

I would agree over landing the upfront approach, which is a proof of concept already, while keeping the design open enough that a “codegen on demand“ mode could be explored as future alternative, the only question is whether the unoptimized code is generated at build time or debug time, and that could eventually be a flag.

1 Like
20

Thanks for sharing that, I hadn’t seen that paper before. There’s a bit of a difference in the implementation, but overall the design looks strikingly similar. From a quick read-through I think two differences are 1) their approach doesn’t include debug information in the optimized module, and 2) they don’t attempt to prevent interprocedural optimizations (which causes them some issues). The mechanics of the optimized/unoptimized code switching seem essentially the same as with our nested-ELF proposal.

2 Likes