Make a StackFrame return immediately

Filipe_Cabecinhas · July 19, 2011, 12:23am

Hi,

Is there a way to mimic gdb’s "return " command?

(gdb) help return

Make selected stack frame return to its caller.
Control remains in the debugger, but when you continue
execution will resume in the frame above the one now selected.
If an argument is given, it is an expression for the value to return.

I’ve been looking at the StackFrame class, but it doesn’t look like it can do that.
If I wanted to implement it, where should I look first? I can get the return address (StackFrame.GetRegisterContext().get().GetReturnAddress(), I think) write it to the PC (StackFrame.ChangePC()), but I have no idea how to get the expression’s result into the return registers/memory. ClangExpression isn’t a big help there, since the result comes to debugger-land.

Regards,

Filipe

Greg_Clayton1 · July 19, 2011, 1:05am

Hi,

Is there a way to mimic gdb's "return <expr>" command?

Not if you want to change the return value.

If you don't want to change the return value you can use "thread step-out". "thread step-out" (which is aliased to "finish"), is context sensitive to the frame you are currently in, so if you run and stop in a frame and then do a "frame select 12", and then to a "thread step-out", you will return to frame 13.

(gdb) help return
Make selected stack frame return to its caller.
Control remains in the debugger, but when you continue
execution will resume in the frame above the one now selected.
If an argument is given, it is an expression for the value to return.

I've been looking at the StackFrame class, but it doesn't look like it can do that.
If I wanted to implement it, where should I look first? I can get the return address (StackFrame.GetRegisterContext().get().GetReturnAddress(), I think) write it to the PC (StackFrame.ChangePC()), but I have no idea how to get the expression's result into the return registers/memory.

LLDB currently doesn't have any real idea of where the return address goes, we currently let the compiler handle all ABI issues by the way we make expressions.

There is another issue where if a function isn't external, the compiler can make a call to this function and how the function returns the value, violate the ABI. In most cases you won't get affected by this, but it would be nice if we knew for sure from the compiler or debug info where the return value is. The old ARM compiler used to inject artificial DW_TAG_variable debug information entries into the DWARF that would tell you the result of functions which has a location that describes the returned value and where it is.

ClangExpression isn't a big help there, since the result comes to debugger-land.

Yep, and even so there is the issue that internal functions can violate the ABI...

FYI: anything ABI related is currently in the ABI plug-ins:

lldb/source/Plugins/ABI/*

The ABI function:

    virtual bool
    ABI::GetReturnValue (Thread &thread,
                    Value &value) const = 0;

Take a look a the ABIMacOSX_i386 and ABIMacOSX_x86_64 versions of this function and see if this does close to what you want. You can also fill in more functionality inside these for types you want it to support. Currently we fill the "value" argument with the result, but we don't fill in the context (See the "void Value::SetContext (ContextType context_type, void *p)" function for details, but the ABI::GetReturnValue functions can be modified to fill in the register context for return values that are returned in registers, and the address (See the "Value::SetValueType (...)" function) if needed.

These functions currently will attempt to extract the return value for a function according to the ABI rules for simple pointer size or less types only (no structs, floats, complex etc). So this might help you for the simple cases. If you were to implement this command you would want to add a new "return" subcommand in the "thread" multi-word command. In the "Execute" function of the new "return" command you would want to evaluate an expression an store the result, set a breakpoint at the return address, install a breakpoint callback and run and hit the breakpoint, then try and instert the expression result into the appropriate location (you would need to modify the "ABI::GetReturnValue(...)" to fill in the "value" param more completely with the location of the return type.

Greg Clayton

Filipe_Cabecinhas · July 22, 2011, 7:35pm

Hi,

Is there a way to mimic gdb’s "return " command?

Not if you want to change the return value.

If you don’t want to change the return value you can use “thread step-out”. “thread step-out” (which is aliased to “finish”), is context sensitive to the frame you are currently in, so if you run and stop in a frame and then do a “frame select 12”, and then to a “thread step-out”, you will return to frame 13.

Yes, that command will run until the frame returns (like gdb’s finish command), but what if we want to return immediately?

(gdb) help return
Make selected stack frame return to its caller.
Control remains in the debugger, but when you continue
execution will resume in the frame above the one now selected.
If an argument is given, it is an expression for the value to return.

I’ve been looking at the StackFrame class, but it doesn’t look like it can do that.
If I wanted to implement it, where should I look first? I can get the return address (StackFrame.GetRegisterContext().get().GetReturnAddress(), I think) write it to the PC (StackFrame.ChangePC()), but I have no idea how to get the expression’s result into the return registers/memory.

LLDB currently doesn’t have any real idea of where the return address goes, we currently let the compiler handle all ABI issues by the way we make expressions.

There is another issue where if a function isn’t external, the compiler can make a call to this function and how the function returns the value, violate the ABI. In most cases you won’t get affected by this, but it would be nice if we knew for sure from the compiler or debug info where the return value is. The old ARM compiler used to inject artificial DW_TAG_variable debug information entries into the DWARF that would tell you the result of functions which has a location that describes the returned value and where it is.

ClangExpression isn’t a big help there, since the result comes to debugger-land.

Yep, and even so there is the issue that internal functions can violate the ABI…

FYI: anything ABI related is currently in the ABI plug-ins:

lldb/source/Plugins/ABI/*

The ABI function:

virtual bool
ABI::GetReturnValue (Thread &thread,
Value &value) const = 0;

Take a look a the ABIMacOSX_i386 and ABIMacOSX_x86_64 versions of this function and see if this does close to what you want. You can also fill in more functionality inside these for types you want it to support. Currently we fill the “value” argument with the result, but we don’t fill in the context (See the “void Value::SetContext (ContextType context_type, void *p)” function for details, but the ABI::GetReturnValue functions can be modified to fill in the register context for return values that are returned in registers, and the address (See the “Value::SetValueType (…)” function) if needed.

These functions currently will attempt to extract the return value for a function according to the ABI rules for simple pointer size or less types only (no structs, floats, complex etc). So this might help you for the simple cases. If you were to implement this command you would want to add a new “return” subcommand in the “thread” multi-word command. In the “Execute” function of the new “return” command you would want to evaluate an expression an store the result, set a breakpoint at the return address, install a breakpoint callback and run and hit the breakpoint, then try and instert the expression result into the appropriate location (you would need to modify the “ABI::GetReturnValue(…)” to fill in the “value” param more completely with the location of the return type.

Greg Clayton

I implemented a thread return command, which works (right now) for integers and pointers, for i386 and x86-64. The command has some caveats (like the ones discussed here), but it works (at least) for a few simple examples in C.

"thread return " returns the result of evaluating that expression. I’m not casting the expression, so there may be some problems (especially when returning floating point types is implemented).

But…
We have no idea if the compiler generated a stackframe for this function or not. We may be returning from two functions, and not one.
We have no ideia if the compiler changed the calling conventions for this code.
We don’t know about the callee-saved registers (I suppose there isn’t a way to find out the epilogue of the function… Especially since it may be mixed with “regular code”)

There are a lot more issues for C++ code (e.g: calling dtors).

I also have some problems updating the state lldb thinks the thread is in.

“register read pc” won’t read the real value
“thread backtrace” doesn’t get updated.

How can I fix that? I am also looking for comments on stuff to fix/improve.

I also have some tests (a directory to put in lldb/test/).

Regards,

Filipe Cabecinhas

P.S: Converting between a ValueObject object and

thread-return.patch (23.4 KB)

thread_return.zip (38.1 KB)

jingham · July 22, 2011, 10:18pm

Filipe,

Thanks for working on this.

Note, gdb's "return" command doesn't do anything to attempt to clean up local objects. It just pops the frames off the stack & returns the value. This, like moving the PC by hand, is one of those commands that you just shouldn't use if you don't know what could go wrong... Anyway, people mostly use this to hack system routines to do something else, in which case you stop at the very beginning and return directly - you haven't gotten far enough to need to do any cleanup.

Seems to me that the sensible thing to do about the return expression is take whatever the user gives you and cast it to the return type of the function you are returning from, if you know it. If you don't then just write it as whatever they have given you, and the user will have to get it right when they write the expression. That's the best you can do in this case.

It also seems to me that much more of this could be generic. The algorithm gdb uses is to get the register context one frame up on the stack, and write that to the real registers, and then throw away the cached stack and remake it from the changed registers. That can all be done generically. Mind there are target specific bits but they are all hidden under the register code. It is the unwinder's job to know what the register state of the functions up on the stack are, you should rely on that.

The only ABI specific bit is that you need to ask the ABI where to store the return value. For things smaller than a register that is easy. For struct returns it can be hard, because the compiler usually passes in a pointer to the save location, but you would have to track the history of that value to know where it is stored at the point you are trying to return from, and that's not trivial. Again, if you are at the beginning of the function you're returning from, this is easy to do.

You can also check whether the block you are in is inlined, and if it is, then all you do is move the PC to the end of the block. You can't really do anything about the return value then, because you can't really tell where it is going, but at least you can return properly from the inlined function.

Hope that helps...

Jim

Filipe_Cabecinhas · July 22, 2011, 10:46pm

Filipe,

Thanks for working on this.

Note, gdb’s “return” command doesn’t do anything to attempt to clean up local objects. It just pops the frames off the stack & returns the value. This, like moving the PC by hand, is one of those commands that you just shouldn’t use if you don’t know what could go wrong… Anyway, people mostly use this to hack system routines to do something else, in which case you stop at the very beginning and return directly - you haven’t gotten far enough to need to do any cleanup.

Seems to me that the sensible thing to do about the return expression is take whatever the user gives you and cast it to the return type of the function you are returning from, if you know it. If you don’t then just write it as whatever they have given you, and the user will have to get it right when they write the expression. That’s the best you can do in this case.

OK.

It also seems to me that much more of this could be generic. The algorithm gdb uses is to get the register context one frame up on the stack, and write that to the real registers, and then throw away the cached stack and remake it from the changed registers. That can all be done generically. Mind there are target specific bits but they are all hidden under the register code. It is the unwinder’s job to know what the register state of the functions up on the stack are, you should rely on that.

The only ABI specific bit is that you need to ask the ABI where to store the return value. For things smaller than a register that is easy. For struct returns it can be hard, because the compiler usually passes in a pointer to the save location, but you would have to track the history of that value to know where it is stored at the point you are trying to return from, and that’s not trivial. Again, if you are at the beginning of the function you’re returning from, this is easy to do.

Ok. You’re suggesting taking the code out of the ABI and putting it on the command (or maybe the StackFrame)? I initially wrote the code thinking calling conventions could vary a lot but, for the returns, they don’t vary that much (and lldb allows us to abstract most of what varies). Or would it be preferred to just contain the code in the command object?

I suppose the command should also be changed to the “frame” command, to mimic gdb (it returns from the current frame, not the bottom-most).

You can also check whether the block you are in is inlined, and if it is, then all you do is move the PC to the end of the block. You can’t really do anything about the return value then, because you can’t really tell where it is going, but at least you can return properly from the inlined function.

Hope that helps…

I will try that one too, thanks.

Regards,

Filipe

jingham · July 22, 2011, 11:23pm

Filipe,

Thanks for working on this.

Note, gdb's "return" command doesn't do anything to attempt to clean up local objects. It just pops the frames off the stack & returns the value. This, like moving the PC by hand, is one of those commands that you just shouldn't use if you don't know what could go wrong... Anyway, people mostly use this to hack system routines to do something else, in which case you stop at the very beginning and return directly - you haven't gotten far enough to need to do any cleanup.

Seems to me that the sensible thing to do about the return expression is take whatever the user gives you and cast it to the return type of the function you are returning from, if you know it. If you don't then just write it as whatever they have given you, and the user will have to get it right when they write the expression. That's the best you can do in this case.

OK.

It also seems to me that much more of this could be generic. The algorithm gdb uses is to get the register context one frame up on the stack, and write that to the real registers, and then throw away the cached stack and remake it from the changed registers. That can all be done generically. Mind there are target specific bits but they are all hidden under the register code. It is the unwinder's job to know what the register state of the functions up on the stack are, you should rely on that.

The only ABI specific bit is that you need to ask the ABI where to store the return value. For things smaller than a register that is easy. For struct returns it can be hard, because the compiler usually passes in a pointer to the save location, but you would have to track the history of that value to know where it is stored at the point you are trying to return from, and that's not trivial. Again, if you are at the beginning of the function you're returning from, this is easy to do.

Ok. You're suggesting taking the code out of the ABI and putting it on the command (or maybe the StackFrame)? I initially wrote the code thinking calling conventions could vary a lot but, for the returns, they don't vary that much (and lldb allows us to abstract most of what varies). Or would it be preferred to just contain the code in the command object?

Yes. This functionality seems generally useful, so the code to do this should go somewhere in the core. Then the command would just be a thin wrapper. StackFrame doesn't seem right to me, since you are operating on the list of frames more than the individual frame. So either StackFrameList, or its owner Thread, seem good places.

I suppose the command should also be changed to the "frame" command, to mimic gdb (it returns from the current frame, not the bottom-most).

Yes, that makes sense, you would either return from the currently selected frame, or from a frame given explicitly in the command. Note in general in lldb we try to avoid positional arguments, so I would do:

frame return --frame <FRAME NUMBER> <expression>

BTW, you can also make LLDB commands "RAW" commands, which means that everything after the options is passed unparsed to the command. That's very convenient for commands that take an expression as their argument, since then you don't end up having to backslash yourself to death. See the "expression" command for an example of this.

Oh, yeah, another thing, though very trivial, since you defined your command with:

eFlagProcessMustBeLaunched | eFlagProcessMustBePaused

you don't have to check whether the process & target are valid, the command interpreter shouldn't call you if they are not.

Again, thanks for working on this!

Jim

Filipe_Cabecinhas · July 22, 2011, 11:25pm

Thanks for the help and advice. I will ping back the list with more stuff when I’m done.

Regards,

Filipe

Filipe_Cabecinhas · July 25, 2011, 10:08pm

Hi all,

I’ve refactored the code to put most of the work on the StackFrameList class, but I still have one problem:
The state of execution isn’t being updated.

If I try to print the pc register after returning, it will just give me the old value (still in the function).
The first thread step-over command after returning from a function isn’t working, too. But that may have to do with:
I can’t update the stackframe list. thread backtrace always puts me inside the function.

These three problems may be interconnected. I’ll try to debug further.

I also can’t do what Jim suggested. I got the Block reference, from the frame. I can see if it’s inlined and its size, but can’t tell where it starts/ends.

I’ll send two patches: They’re the same except for the implemented command.

In one, the command is “thread return”, in the other it is “frame return”.

My problem is… Except for “frame select”, the frame commands only work on the current frame (not on other frames), so I would see “frame return” as return only from the current frame. While “thread return” could return from any frame (defaulting for the current thread). What do you think?

I’m also sending a test directory, using the “thread return” variation.

Regards,

Filipe Cabecinhas

frame-return.patch (30.2 KB)

thread_return.zip (6.68 KB)

thread-return.patch (30.7 KB)

Filipe_Cabecinhas · July 27, 2011, 12:29am

Hi all,

Multi-level returns were not working as intended, as I would write to the frame’s RegisterContext (yielding an error), instead of writing to the live RegisterContext. I have fixed that and created another test-case for it.

I still have the problem of not rebuilding the StackFrameList, and the first “thread step-over” command right after a “frame return” is yielding an error. Subsequent commands work fine.

Regards,

Filipe

frame_return.zip (4.15 KB)

frame-return.patch (30.6 KB)

Enrico_Granata · July 27, 2011, 12:36am

Hi Filipe,
thanks for all the hard work you are doing on LLDB.

While I have not really delved into your patch, I think a good way to find out why you get this persistent error on stepping-over after your new command would be to set a breakpoint in CommandObjectThreadStepWithTypeAndScope::Execute

The way I usually do this is creating an instance of LLDB debugging my inferior process (a test program in your case), and then in another Terminal window, opening a second LLDB and attaching it to the first instance (you can use the process attach -p command to do so).
Then start to play with the first LLDB, and when you get to the point where you would type thread step-over and get in trouble, just set a breakpoint into the CommandObjectThreadStepWithTypeAndScope::Execute call in the second LLDB instance.

Now, you can step through the command’s code and check where it’s failing exactly (and hopefully figure out why).

Sincerely,

Enrico Granata

✆ 408.974.5572 | egranata@.com

jingham · July 27, 2011, 12:42am

Did you try calling ClearStackFrames() in Thread::ReturnFromSelectedFrame (after you call the ReturnFrom call, of course)? That should flush all the cached frames, and the next time somebody tries to access the frames for that stack, they will be automatically recomputed from the live register state you've written. That should also make the stepping after the return work correctly.

Jim

Filipe_Cabecinhas · July 27, 2011, 1:08am

Yes, I tried that. I still get the same problem.

Another weird thing is the “thread step-over” problem. It seems, somehow, the thread (the only one) is just not selected anymore. But the next time I try to run the same command, it is selected (And I can’t find out where it is getting unselected…). I will try to continue the debugging process.

If I run the command “thread select 1” right after the “frame return” command, I don’t get the error, since I manually selected the thread.

Any clues?

Regards,

Filipe

Filipe_Cabecinhas · July 27, 2011, 5:27pm

Hi,

I did that, and tried watching the thread list, to see when it changed… Couldn’t find anything. I’ll continue searching the code and debugging to see where the selected thread may be changing. Or is it something that shouldn’t happen?

I will also try this with a Cocoa program, to check the objective-c returns and if having more threads change anything.

Regards,

Filipe Cabecinhas

Filipe_Cabecinhas · July 29, 2011, 9:10pm

Hi,

Is it normal for these two threads to differ?

exe_ctx = m_interpreter.GetExecutionContext();

exe_ctx.thread: 0x107900610, exe_ctx.process->GetThreadList().GetSelectedThread(): 0x0

Also, if we only have one thread, shouldn’t that thread always be selected? Somehow the ThreadList.m_selected_thread is being set to LLDB_INVALID_THREADID right after I run my command.

Regards,

Filipe

Greg_Clayton1 · July 29, 2011, 9:52pm

You should always be looking at the m_interpreter for the correct execution context to use. The selected thread is for use in the command interpreter:

(lldb) thread select 12
(lldb) <command>

Now when <command> is executed, it might cause the process to run (as is happening in your case), and the selected thread might change or get invalidated.

In this case the selected thread (exe_ctx.process->GetThreadList().GetSelectedThread()) matches the interpreter thread (m_interpreter.GetExecutionContext().thread), but this isn't always true.

So at the start of your command execution, you should remember the thread (and the frame if you need that) that was selected if that is important to you by storing it away (copy the thread shared pointer) in your CommandObject subclass. Later you might also need to verify that your thread is still alive and well by checking for the thread in the process using the

ThreadSP
ThreadList::FindThreadByIndexID (uint32_t index_id, bool can_update);

The index id is a unique thread index that will continue to increase as new threads come and go. This guarantees that if a system can re-use thread ids (not thread index ids, but thread ids or tid), that if a thread comes and goes that we can track that we were actually trying to step on the correct thread.