DW_OP_deref handling

Hi Greg,
I have this issue that I suspect is a bug in lldb’s handling of DW_OP_deref.
I wasn’t able to craft an easy C++ testcase, so I’ll start from my
motivating (swift) example.

Given the following code:

func use<T>(_ t : T) {}
func single<T>(_ t : T) {
  let x = t
  use(x)
}
let s = "hello"
single(s)

The IR emitted for the local binding `x` in `single<T>` is an alloca
containing the address of `x`. Hence, the DWARF, correctly contains a
DW_OP_deref expression, i.e.:

0x000000da: TAG_variable [9]
                             AT_location( fbreg -16, deref )
                             AT_name( "x" )
                             AT_decl_file(
"/Users/dcci/work/swift/build/Ninja-RelWithDebInfoAssert+stdlib-RelWithDebInfo/swift-macosx-x86_64/bin/pat.swift"
)
                             AT_decl_line( 4 )
                             AT_type( {0x00000204} ( $sxD ) )

When I debug this with lldb, I expect lldb to print the value that’s
pointed by %rbp - 16,
i.e.
(lldb) register read $rbp
     rbp = 0x00007ffeefbff860

(lldb) mem read 0x00007ffeefbff850
0x7ffeefbff850: 10 f8 bf ef fe 7f 00 00 b8 32 4d 00 01 00 00 00
.???...?2M.....
0x7ffeefbff860: c0 f8 bf ef fe 7f 00 00 64 09 00 00 01 00 00 00
???...d.......

So, the value that I expected to be printed is 0x7ffeefbff810.

What I instead see in the debugger is:
(T) x = <read memory from 0xe500000000000000 failed (0 of 16 bytes read)>

The value `0xe500000000000000` happens to be the value that’s pointed
by the address 0x7ffeefbff810, that is:

(lldb) mem read 0x7ffeefbff810
0x7ffeefbff810: 00 00 00 00 00 00 00 e5 68 65 6c 6c 6f 00 00 00
.......?hello...
0x7ffeefbff820: 05 00 00 00 00 00 00 00 40 32 4d 00 01 00 00 00
........@2M.....

Is this an expected behavior? I did expect DW_OP_deref to just do a
single dereference and not two. It looks like when we call
`UpdateValue()` in lldb we do have the correct value in `m_value`, but
GetPointerValue() ends up printing the value that’s *pointed by*
what’s in `m_value`, i.e.

    m_integer = {
      U = {
        VAL = 0x00007ffeefbff810
        pVal = 0x00007ffeefbff810
      }

as this is a LoadAddress. Greg, as you wrote this code (and touched it
last), what do you expect to be the correct semantic here?

Removing a dereference from DW_OP_deref in the DWARFASTParser works
around the issue, but I’m fairly sure it’s the incorrect fix. Are we
maybe setting the value type incorrectly?

Best,

The issue is DW_OP_deref dereferences a pointer and pushes it onto the stack. The code currently, for a load address that is on top of the stack does:

Value::ValueType value_type = stack.back().GetValueType();
switch (value_type) {

case Value::eValueTypeLoadAddress:
if (exe_ctx) {
if (process) {
lldb::addr_t pointer_addr = stack.back().GetScalar().ULongLong(LLDB_INVALID_ADDRESS);
Status error;
lldb::addr_t pointer_value = process->ReadPointerFromMemory(pointer_addr, error);
if (pointer_value != LLDB_INVALID_ADDRESS) {
stack.back().GetScalar() = pointer_value;
stack.back().ClearContext();
} else {
return false;
}

Here we just use “stack.back()” to access the item at the top of the expression value stack. The top value has a Value::ValueType of Value::eValueTypeLoadAddress. We don’t reset this value to be Value::eValueTypeScalar, but we probably should and that will probably fix this issue.

DWARF5 finally added the ability to track what each value means on the expression stack. Prior to DWARF 5, we had no idea what each entry on the expression value stack was (file address, load address (Value::eValueTypeLoadAddress), plain value (Value::eValueTypeScalar). We have tracked this info for a while now, but the DWARF5 spec is much more specific on how things should be treated. From the spec:

DW_OP_deref

The DW_OP_deref operation pops the top stack entry and treats it as an address. The popped value must have an integral type. The value retrieved from that address is pushed, and has the generic type. The size of the data retrieved from the dereferenced address is the size of an address on the target machine.

No the “The value retrieved from that address is pushed, and has the generic type.” statement. In LLDB this means the value’s ValueType should be set to Value::eValueTypeScalar.

So try modifying the code with stack.back().SetValueType(Value::eValueTypeScalar) after the stack.back().ClearContext():

if (pointer_value != LLDB_INVALID_ADDRESS) {
stack.back().GetScalar() = pointer_value;
stack.back().ClearContext();
**stack.back().**SetValueType(Value::eValueTypeScalar);
} else {
return false;
}

That might fix things.

The way the expression ends up right now we end up with an expression stack saying that the value is a load address which means we must read the value from memory in order to find it. This would be the right thing to do if your location expression was:

0x000000da: TAG_variable [9]
AT_location( fbreg -16 )
AT_name( “x” )
AT_decl_file(…)
AT_decl_line( 4 )
AT_type( {0x00000204} ( $sxD ) )

Note I removed the “deref” from the location expression. In this case the variable would need to be read from memory. If the value was to be in a register, then the location would be:

0x000000da: TAG_variable [9]
AT_location( reg16 )
AT_name( “x” )
AT_decl_file(…)
AT_decl_line( 4 )
AT_type( {0x00000204} ( $sxD ) )

Then the value would be in register 16 itself (not dereferenced).

Hope this all makes sense. If you have any questions let me know.

Greg

Thanks. I tend to agree this is the right thing to do, but it happens
to break many tests in the debugger.

Failing Tests (26):

    lldb-Suite :: functionalities/unwind/sigtramp/TestSigtrampUnwind.py
    lldb-Suite :: lang/c/blocks/TestBlocks.py
    lldb-Suite :: lang/swift/address_of/TestSwiftAddressOf.py
    lldb-Suite ::
lang/swift/expression/class_constrained_protocol/TestClassConstrainedProtocol.py
    lldb-Suite ::
lang/swift/expression/exclusivity_suppression/TestExclusivitySuppression.py
    lldb-Suite ::
lang/swift/expression/optional_amibiguity/TestOptionalAmbiguity.py
    lldb-Suite :: lang/swift/expression/scopes/TestExpressionScopes.py
    lldb-Suite :: lang/swift/expression/weak_self/TestWeakSelf.py
    lldb-Suite ::
lang/swift/foundation_value_types/data/TestSwiftFoundationTypeData.py
    lldb-Suite ::
lang/swift/foundation_value_types/date/TestSwiftFoundationTypeDate.py
    lldb-Suite ::
lang/swift/foundation_value_types/indexpath/TestSwiftFoundationTypeIndexPath.py
    lldb-Suite ::
lang/swift/foundation_value_types/measurement/TestSwiftFoundationTypeMeasurement.py
    lldb-Suite ::
lang/swift/foundation_value_types/notification/TestSwiftFoundationTypeNotification.py
    lldb-Suite ::
lang/swift/foundation_value_types/url/TestSwiftFoundationTypeURL.py
    lldb-Suite ::
lang/swift/foundation_value_types/uuid/TestSwiftFoundationTypeUUID.py
    lldb-Suite :: lang/swift/generic_tuple/TestSwiftGenericTuple.py
    lldb-Suite :: lang/swift/po/sys_types/TestSwiftPOSysTypes.py
    lldb-Suite :: lang/swift/resilience/TestResilience.py
    lldb-Suite ::
lang/swift/variables/generic_struct_debug_info/generic_flatmap/TestSwiftGenericStructDebugInfoGenericFlatMap.py
    lldb-Suite :: lang/swift/variables/inout/TestInOutVariables.py
    lldb-Suite :: lang/swift/variables/protocol/TestSwiftProtocolTypes.py
    lldb-Suite ::
lang/swift/variables/uninitialized/TestSwiftUninitializedVariable.py
    lldb-Suite :: types/TestFloatTypes.py
    lldb-Suite :: types/TestFloatTypesExpr.py
    lldb-Suite :: types/TestIntegerTypes.py
    lldb-Suite :: types/TestIntegerTypesExpr.py

  Expected Passes : 1488
  Unsupported Tests : 27
  Unexpected Failures: 26

My guess here is that there are even more hacks to work around this
ambiguity of interpretation. The good thing is that many of the
failure are in the swift support, so I can probably navigate through
them relatively quickly. I'll analyze them one by one and maybe
consult you in case there's something that doesn't make se use here.
(There are several C/C++ failure here, but I'll think about those
later) :slight_smile:

I am happy to look at any DWARF expressions you have and help figure out where the change needs to go or how the expression need to be fixed.

Greg

Thank you for your help, it's greatly appreciated. hopefully we'll
have a more precise debugger after this effort :slight_smile:

Now that we have a more precise specification in DWARF5, we can. Before that, it was all up to interpretation of each compiler engineer as to what made sense for the location expressions...

Thanks for working on this.

Greg,

I'd like to summarize my own understanding of how this works — could you take a look and correct me where I'm wrong?

- The only way to push a file address onto the DWARF stack is a DW_OP_addr.

The decision of whether a value pushed onto the DWARF stack is a scalar or a load address depends on the location kind (cf. DWARF 5, section 2.6 "Location Descriptions"):
- A register location description (DW_OP_reg.*) reads the register contents and pushes a scalar.
- An implicit location description (.* (DW_OP_implicit_.*|DW_OP_stack_value) yields a scalar after evaluating the expression.
- A memory location description (anything else, such as DW_OP_breg) yields a load address.
(- composite locations, like DW_OP_piece are handled according to these rules for each piece)

Practically speaking, I think this means that a DW_OP_(f)breg always turns into a load address (as it can only appear in an implicit or a memory location), and a DW_OP_reg always. turns into a scalar.

Is that what LLDB is doing, and if not, could that explain at least some of the failures that Davide is seeing?

-- adrian

To answer my own question: I think that the patch to set the address type to scalar after interpreting a DW_OP_deref is just wrong, because of the above :slight_smile:

-- adrian

Follow up. This turned out to be a bug in the swift specific support,
so I don't think the DWARF parser code is wrong here.

Thanks!

Correct!

Summarizing:
- DW_OP_addr == file address
- DW_OP_(f)breg == load address
- DW_OP_reg == scalar
- DW_OP_deref(x) == needs to be fixed to change to scalar, currently leaves it as load addr
- DW_OP_const == scalar (unless DW_OP_piece is used)

The failure Davide is seeing is there is a load address on the stack, and after the deref, it gets pushed the correct value but leaves it as a load addr.

When any expression is done being evaluated, we look at the type of that value that is left at the top of the stack.

If it is a load addr, then we read from memory
if it is a scalar, it is the value itself

Issue Davide is running into is the value is correct, but the type is still load addr incorrectly.

Greg