Expressions aren't evaluated with respect to target's endianness

Hello people again,

Still working on that LLDB stub controlling a big-endian target. I'm working on evaluating expressions. I just realized that lldb doesn't respect the target byte order when it saves expression results.

This expression:

    expr int $x = 4;

causes lldb to allocate memory in the inferior process then write 04 00 00 00 to it, which is little-endian for 4, even though my target is big endian.

The ClangUserExpression was initialized with the correct byte order, but things seem to go wrong with IRInterpreter::ResolveConstant, which calls APInt::getRawData(), returning a pointer to an uint64_t in host byte order. It then calls m_memory_map.WriteMemory, casting the uint64_t* into a uint8_t*, and the rest is history.

I guessed from the previous endianness bugs I reported that it could be fixed with a SetByteOrder call, but turns out that there's no DataExtractor involved. It could be swapped in IRMemoryMap::ResolveConstant, but I'm not sure that's the correct thing to do.



thanks for looking at this. I had a quick look through the IRInterpreter.

The implementation of Instruction::Load and Instruction::Store are probably okay, since they’re just moving data around.
It also appears that when we read data into Scalars, we read it correctly, because we’re using EvaluateValue, which uses a properly-initialized DataExtractor. Writing data out of Scalars is also okay, because it uses GetAsMemoryData, which is parameterized with a byte order.

As a result, I think the problem is localized to ResolveConstant. Your intuition was probably right: the data coming out of ResolveConstantValue needs to be byte-swapped before being written into memory. I’d do that right in ResolveConstant, by using a StreamString (initialized with flags = eBinary and addr_size and byte_order to match the target).

If you can get me a patch I’ll be happy to review it.


Thanks Sean. Here's a patch.


FixExpressionEndianness.diff (1.4 KB)