Seeking clarification and way forward on limited scope variables.

Hello Everyone,

I need to have your thoughts on this.

Consider the following test case –

As always, concerned about the size growth in object files this might produce - though looks like the DWARF spec avoids the worst of this in unoptimized code by using an offset relative to the start of the enclosing scope, so it doesn’t require a relocation in that case.

I have no idea what the LLVM DWARF representation for this would look like - short of making even more fine-grained scopes in the DILexicalScope hierarchy, which sounds really expensive from a memory perspective. That’s really where I worry that the cost to this feature might outweigh the benefit (& might be why no one’s done this in the past) - but data should tell us that. As much as in-tree development is preferred, this might be the sort of thing worth prototyping out of tree first to see if it can be made viable before adding the complexity to the LLVM project proper - but I’m open to ideas/suggestions.

Hi Sourabh,

Thanks for raising this issue. To answer your question, (afaik) there isn’t anyone working on DW_AT_start_scope support in tree. We’re looking for a solution to this problem for Swift debugging, where it’s important not to make a debug location for a variable available until its (guaranteed) initialization is complete.

If at all possible, I’d /much/ rather we use the existing location list machinery to solve this problem. Fundamentally, we’re looking for a way to express when a location for a variable becomes available, and location lists give us that already.

To test this out, I took the IR for your test case, replaced all calls to “dbg.declare(…)” with “dbg.value(…, DW_OP_deref)”, and compiled with -g -O0 -mllvm -fast-isel=false (apparently FastISel doesn’t know what to do with frame-index dbg.values, but this is simple to fix). This seemed to work pretty well, and the DWARF looked legit:

0x0000005f: DW_TAG_variable
DW_AT_location (DW_OP_fbreg -20)
DW_AT_name ("Local")

0x0000006d: DW_TAG_lexical_block
DW_AT_low_pc (0x0000000100000f4f)
DW_AT_high_pc (0x0000000100000f84)

0x0000007a: DW_TAG_variable
DW_AT_location (0x00000000
[0x0000000100000f63, 0x0000000100000f8c): DW_OP_breg6 RBP-24)
DW_AT_name ("Local”)

I did find one lldb bug where it didn’t know to look up a variable in the parent scope when stopped at your second printf() call (line 6). But that’s a general bug: we’d have to fix it even if we used DW_AT_start_scope.

The upshot of sticking to location lists is that fixing any bugs we find improves optimized debugging workflows. And Swift -Onone debugging workflows as well, since Swift runs certain mandatory optimizations which can make the DWARF at -Onone fairly complex.

best,
vedant

Hi Sourabh,

Thanks for raising this issue. To answer your question, (afaik) there isn’t anyone working on DW_AT_start_scope support in tree. We’re looking for a solution to this problem for Swift debugging, where it’s important not to make a debug location for a variable available until its (guaranteed) initialization is complete.

If at all possible, I’d /much/ rather we use the existing location list machinery to solve this problem. Fundamentally, we’re looking for a way to express when a location for a variable becomes available, and location lists give us that already.

To test this out, I took the IR for your test case, replaced all calls to “dbg.declare(…)” with “dbg.value(…, DW_OP_deref)”, and compiled with -g -O0 -mllvm -fast-isel=false (apparently FastISel doesn’t know what to do with frame-index dbg.values, but this is simple to fix). This seemed to work pretty well, and the DWARF looked legit:

0x0000005f: DW_TAG_variable
DW_AT_location (DW_OP_fbreg -20)
DW_AT_name ("Local")

0x0000006d: DW_TAG_lexical_block
DW_AT_low_pc (0x0000000100000f4f)
DW_AT_high_pc (0x0000000100000f84)

0x0000007a: DW_TAG_variable
DW_AT_location (0x00000000
[0x0000000100000f63, 0x0000000100000f8c): DW_OP_breg6 RBP-24)
DW_AT_name ("Local”)

I did find one lldb bug where it didn’t know to look up a variable in the parent scope when stopped at your second printf() call (line 6). But that’s a general bug: we’d have to fix it even if we used DW_AT_start_scope.

The upshot of sticking to location lists is that fixing any bugs we find improves optimized debugging workflows. And Swift -Onone debugging workflows as well, since Swift runs certain mandatory optimizations which can make the DWARF at -Onone fairly complex.

I think these are different problems, and that location lists can’t be used to express when the scope of a variable starts - Sourabh’s example at the end shows why: that the debugger wouldn’t search outside the scope, it would instead (correctly) report the variable’s value as unavailable.

“I did find one lldb bug where it didn’t know to look up a variable in the parent scope when stopped at your second printf() call (line 6).”

I don’t think that’s not a bug, though/that change would be incorrect in general (& no way to differentiate the correct and incorrect places without scope_start), for example:

void f1();
int i;

void f2() {

int i = 3;

f1();

int j;

i = j;

f1();

}

If you search into outer scopes whenever a location list doesn’t cover an address - then at the second call to ‘f1’ the debugger would interpret ‘i’ as referring to global ‘i’ when it should be referring to local ‘i’ just with no known value. Printing global ‘i’ could be quite confusing/misleading.

Hi Vedant,

Thanks for quick response.

If at all possible, I’d /much/ rather we use the existing location list machinery to solve this problem. Fundamentally, we’re looking for a way to express when a location for a variable becomes available, and location lists give us that already.

We also thought of location list as a immediate solution here, but AFAIK location list is not emitted(or constructed) in at -O0(no optimizations), with clang and gcc.

To test this out, I took the IR for your test case, replaced all calls to “dbg.declare(…)” with “dbg.value(…, DW_OP_deref)”, and compiled with -g -O0 -mllvm -fast-isel=false (apparently FastISel doesn’t know what to do with frame-index dbg.values, but this is simple to fix). This seemed to work pretty well, and the DWARF looked legit: …

In our case, we also experimented with this, but as in case of DW_AT_start_case debugger(in our case GDB) is also modified to look for variable in outer scope if PC is not in range of list. Currently GDB works as, if PC is in range fetch that variable other wise report if as optimized out.

I did find one lldb bug where it didn’t know to look up a variable in the parent scope when stopped at your second printf() call (line 6). But that’s a general bug: we’d have to fix it even if we used DW_AT_start_scope.

Yea, I mean it shows two variable at line 6 name Local.

Process 16841 launched: ‘/home/sourabh/work/C++/a.out’ (x86_64)
6
Process 16841 stopped

  • thread #1, name = ‘a.out’, stop reason = breakpoint 1.1
    frame #0: 0x00000000002016dd a.out`main(Argc=1, Argv=0x00007fffffffe4a8) at MainScope.c:6:16
    3 printf(“%d\n”,Local);
    4
    5 {
    → 6 printf(“%d\n”,Local);
    7 int Local = 7;
    8 printf(“%d\n”,Local);
    9 }
    (lldb) frame variable
    (int) Argc = 1
    (char **) Argv = 0x00007fffffffe4a8
    (int) Local = 6
    (int) Local =

Thank You,
Sourabh Singh Tomar.

Thanks David for your example demonstrating why location list won’t be the best fit here, Surprisingly your response came just after(or before) I sent mine!

Just wanted to drop an additional note here WRT lldb log shared previously –
that behavior is when, Local var is having location list, whilst in normal case(no location list for Local var inside Lex Block) also LLDB shows 2 variables at Line No. 6, But the second one contains (as expected)Garabage value.
I think it’s a bug, since we’re inside Lex Block it should not show variable from outer scope(or block) ??

(lldb) frame variable
(int) Argc = 1
(char **) Argv = 0x00007fffffffe4a8
(int) Local = 6
(int) Local = 2102704

Thank You,
Sourabh

Thanks David and Sourabh for your comments. I filed https://bugs.llvm.org/show_bug.cgi?id=45564 to track the lldb bug. I see now that DW_AT_start_case provides some information not captured by location lists, and withdraw my objection.

vedant

Hi Everyone,

We’ve reached to a working prototype (With some caveats mentioned at the end) for this problem, I need to have your opinion on this before we move further WRT.

  • @David BlaikieI have no idea what the LLVM DWARF representation for this would look like - short of making even more fine-grained scopes in the DILexicalScope hierarchy, which sounds really expensive from a memory perspective. That’s really where I worry that the cost to this feature might outweigh the benefit (& might be why no one’s done this in the past) - but data should tell us that. As much as in-tree development is preferred, this might be the sort of thing worth prototyping out of tree first to see if it can be made viable before adding the complexity to the LLVM project proper - but I’m open to ideas/suggestions.

Problem breakdown:

End representation at (Object or ASM level):

  • Need a label(Symbol) to extract exact location(Or address at which the variable defined(or allocated)), for which an Machine Instruction is needed(in this case DBG_VALUE leveraged).
  • Take the difference of this label(address) and low_pc of the Lexical Block, synthesize the “DW_AT_start_scope” using this difference(Offset) attach this attribute to the Scoped variable.

Implementation LLVM-to-DWARF:

  • Implementation strategy is a bit conservative in a sense, I didn’t introduce any new metadata or new instruction(IR or MachineInstr).

  • For the unoptimized case, this variable will be stack allocated(with Frame Index per alloca instruction) and dbg.declare is used.

  • Lower this dbg.declare(associated with the variable of interest) to generate a DBG_VALUE instruction for this dbg.declare intrinsic(in both FastISel and SelectionDAGISel Instruction Selection) Distinguishing with other local variables(if present) is accomplished based on the premises of enclosing scope(i.e Lexical Block).

  • Request a label(Symbol) before this instruction(DBG_VALUE). This label will be use later for associating the variable with it and calculating offset to be inserted as DW_AT_start_scope.

  • Extend DbgVariable to contain 2 symbols

  • VarSym → Symbol associated with this variable

  • ScopeBeginSym → Symbol associated with the enclosing Lexical Scope LowPC- When constructing actual DIE and DW_AT_start_scope use these symbols difference to emit as offset within the variable DIE. Again distinguishing with other variables is accomplished using above mentioned simple check.

Resultant DWARF after this for the test case:

0x0000006d: DW_TAG_lexical_block

DW_AT_low_pc (0x00000000002016d1)

DW_AT_high_pc (0x0000000000201729)

0x0000007a: DW_TAG_variable

DW_AT_location (0x00000000:

[0x00000000002016e8, 0x0000000000201731): DW_OP_breg6 RBP-24)

DW_AT_name (“Local”)

DW_AT_decl_file (“test.c”)

DW_AT_decl_line (7)

DW_AT_type (0x0000008b “int”)

DW_AT_start_scope (0x00000017)

Summarizing results:

  • Debugging behavior as expected(of course with a small change in GDB(as mentioned in very first mail)).
  • No regressions found on trunk!
  • Cost to benefit: Compile time costs and debug info size increment not yet evaluated.

Concerns still need to be addressed:

  • DBG_VALUE instructions in most cases end up creating location lists(due to their special handling), unfortunately in this case too apart from “DW_AT_start_scope” Location lists are also

getting attached to the variable. Is this Okay ?

Could you guys please let us know your thoughts/comments on the overall approach taken here.

Note:

This sort of bug(improvement based on DW_AT_start_scope) is also reported(By someone else, not me) in GCC here:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93844

Thank You!

Sourabh.

Gentle Ping!

Might be easier to discuss with concrete code and test cases - perhaps
you could post them to phab or a github branch or the like?

(specifically it's probably easier (for me, at least) to understand
the proposal by being able to see specific/actual IR, how
optimizations need to handle it, etc)