Same-Process Debugging

I’ve been looking into better leveraging LLDB for debugging it’s own process, effectively allowing you to inspect your own program programmatically with a debugger-centric view.

I already built a functioning API for introspecting C programs using LLDB called Ruminate, with an accompanying paper written about it, but it has to jump through some rather expensive IPC hoops in order to do so.

After talking with Todd Fiala, he suggested writing an implementation of NativeProcessProtocol and NativeThreadProtocol that inspects the local process. This would allow same-process debugging without the expensive IPC that Ruminate has.

Basically I want to be able to use LLDB as a high-level API to inspect your own process, with all of the details of debug symbols, stack traversal, etc handled for you.

Thoughts?

After talking with Todd Fiala, he suggested writing an implementation of NativeProcessProtocol and NativeThreadProtocol that inspects the local process. This would allow same-process debugging without the expensive IPC that Ruminate has.

The idea here was that parts of lldb that currently depend on Process/Thread, and may later be adapted to work with NativeProcessProtocol and NativeThreadProtocol, would still be able to work on a local process introspectively if there was a NativeProcessProtocol/NativeThreadProtocol that operated on itself. I was thinking of it more as a way to enable other parts of the lldb machinery within the local process space.

-Todd

Right now NativeProcessProtocol and NativeThreadProtocol are for debugging other processes. We also want the ability to just look at other processes without debugging them (like a "sample" command that would start and stop a process and be able to look at it, backtrace it, but do nothing more than suspend and resume it. We could build this ability into NativeProcessProtocol and NativeThreadProtocol, but we currently assume that NativeProcessProtocol and NativeThreadProtocol will attach to and debug this other process.

The lldb_private::Host layer can do "host" kind of things, so there could be functions added to the host layer if you want to inspect the current process that contains LLDB.

More comments below.

I've been looking into better leveraging LLDB for debugging it's own process, effectively allowing you to inspect your own program programmatically with a debugger-centric view.

I already built a functioning API for introspecting C programs using LLDB called Ruminate, with an accompanying paper written about it, but it has to jump through some rather expensive IPC hoops in order to do so.

Is this IPC just the debugging of the other process? Or is it more than that?

After talking with Todd Fiala, he suggested writing an implementation of NativeProcessProtocol and NativeThreadProtocol that inspects the local process. This would allow same-process debugging without the expensive IPC that Ruminate has.

What is the IPC needed for? Getting the backtraces? What exactly are you looking to eliminate?

Basically I want to be able to use LLDB as a high-level API to inspect your own process, with all of the details of debug symbols, stack traversal, etc handled for you.

Thoughts?

We currently don't have the "attach and inspect only" of a process, and we currently can't debug ourselves. So the main question is do we try to build this into NativeProcessProtocol and NativeThreadProtocol, or do we make other classes for them.

Greg

Hey Russell,

It might be worth having you sketch out what would be an ideal way for you to make use of the functionality of LLDB (i.e. without the expensive IPC). We might then be able to brainstorm how you can achieve your goals from what we have now, which might have nothing to do with the Native* classes.

-Todd

Right now NativeProcessProtocol and NativeThreadProtocol are for debugging other processes. We also want the ability to just look at other processes without debugging them (like a “sample” command that would start and stop a process and be able to look at it, backtrace it, but do nothing more than suspend and resume it. We could build this ability into NativeProcessProtocol and NativeThreadProtocol, but we currently assume that NativeProcessProtocol and NativeThreadProtocol will attach to and debug this other process.

My implementation of NativeProcessProtocol does nothing when you try to attach to a process.

The lldb_private::Host layer can do “host” kind of things, so there could be functions added to the host layer if you want to inspect the current process that contains LLDB.

I hadn’t looked at lldb_private::Host. I’ll have a look at it and get back to you. NativeProcessProtocol seems like the right layer to add this kind of change to though, as that allows lldb to conceptually continue debugging a remote process, that remote process just happens to be itself.

More comments below.

I’ve been looking into better leveraging LLDB for debugging it’s own process, effectively allowing you to inspect your own program programmatically with a debugger-centric view.

I already built a functioning API for introspecting C programs using LLDB called Ruminate, with an accompanying paper written about it, but it has to jump through some rather expensive IPC hoops in order to do so.

Is this IPC just the debugging of the other process? Or is it more than that?

It’s more than that. When you initialize Ruminate in your own process, it fork()s a process which initializes and controls lldb. The debugger process also starts an RPC server exporting an API to control LLDB. The debugee (or parent process) connects to this RPC server, thereby allowing it to debug itself, proxied via the child process. The startup is actually a bit more expensive than this, but this is the general workflow.

After talking with Todd Fiala, he suggested writing an implementation of NativeProcessProtocol and NativeThreadProtocol that inspects the local process. This would allow same-process debugging without the expensive IPC that Ruminate has.

What is the IPC needed for? Getting the backtraces? What exactly are you looking to eliminate?

Well, for the ptrace() IPC, I have lldb stop the debugee for getting array members, function argument types, enum members and enumerating the members of an sbtype (struct members). See getMembers in Ruminate. It’s been awhile since I wrote that code, but if I remember correctly, the LLDB’s API didn’t allow me to perform these operations without stopping the debugee.

For the RPCs, well all of the information about the debugee is in the debugger process, so even operations that don’t require stopping the debugee involve an RPC.

I’d also like to be able to get rid of this which is disgusting and should die in a fire, but is necessary currently to get information about an array for which you have no value (aka, can’t go through SBValue).

Basically I want to be able to use LLDB as a high-level API to inspect your own process, with all of the details of debug symbols, stack traversal, etc handled for you.

Thoughts?

We currently don’t have the “attach and inspect only” of a process, and we currently can’t debug ourselves. So the main question is do we try to build this into NativeProcessProtocol and NativeThreadProtocol, or do we make other classes for them.

It seemed to make sense to me to implement NativeProcessProtocol (why detailed already above), but if you think there’s a better way, I’m all ears.

As for “attach and inspect only”, I wasn’t really treating that as a separate feature. I manipulate lldb to work like that by changing the signal disposition to pass all signals through (hence my previous patch to add support for changing the signal disposition). Granted, I have to stop the debugee in some places, but my efforts were going to move towards making those stops unnecessary once I had a working local Native*Protocol implementation.

Re: Todd’s response which arrived while I was writing this email:

I think ideally you’d be able to use most of the SB API on your own process, potentially even stopping and setting breakpoints on other threads. Also, you’d be able to do the ruminate-necessary operations of enumerating the type system - enumerating fields of structs, arguments, functions, etc. Does that make sense? I hope I’m getting my thoughts across clearly.

Oh, and backtraces too.

Greg,

I looked at Host, and I’m not sure it’s the best place for these changes. There’s quite a few methods there which will simply call into the already extant implementations for whatever host you’re on. Of the functions that would need to behave differently when debugging yourself, take Backtrace for example, the logic of how to traverse the stack should be shared while the logic of how to read memory will differ.

I think the model whereby lldb treats itself as just another process for most purposes makes sense, and using a NativeProcessProtocol implementation that inspects local memory etc. seems to make sense to me.

Also, I’d made some responses to your other comments in my previous email. What do you think?

Thanks,
Russ Harmon

Ping