How to debug LLDB source codes?

Subprojects LLDB
1

Hi guys, I 'm very interested in the detatiled implementation of LLDB sorce codes, but it’s not easy to use lldb debug lldb source codes itself, and I know the most key component of lldb is called by lldb-server, so I use the following way to do debugging but the breakpoint can not be triggered, so can anybody help me explain why the breakpoint cannot be triggered and provide one method to do lldb source code debugging? Thanks very much. And the following method is what I have used now(and it has problem as what I say above):

simple.c
#include <stdio.h>
void do_stuff(int my_arg)
{
    int my_local = my_arg + 2;
    int i;
    for (i = 0; i < my_local; ++i)
        printf("i = %d\n", i);
}
int main()
{
    do_stuff(2);
    return 0;
}

Step One: clang -O0 simple.c -g -o simple.out
Step Two: lldb – lldb-server g localhost:1234 ./simple.out
In step two, I want to debug lldb-server and set one breakpoint in function InvokeCallback of
Breakpoint.cpp, and I know lldb-server will exexute this function if client set breakpoint and run
the binary.
Step Three: Run lldb and input gdb-remote 1234 to communicate with lldb-server
The running result of lldb-server is:
image

The breakpoint I set in lldb-server is not triggered as what we can see in the picture.

2

First, the way you are debugging lldb-server is fine, and I know it’s not easy to figure that out. The problem is your’re trying to break on a function that it will never run in lldb-server.

The boundary between what goes on in lldb-server and lldb (lldb client as I sometimes call it) is not clear here. InvokeCallback is something that lldb will run, not lldb-server.

How exactly those symbols end up in the lldb-server binary, I’m not sure. It may be that both link to liblldb. I want to say lldb does this as a shared library and lldb-server as a static library (for easier copying of the server to a remote system), but it’s been a while since I looked at that.

If I choose something that is clearly used by lldb-server (well, clearly once you know the source layouts and such):

$ ./bin/lldb -- ./bin/lldb-server g localhost:1234 /tmp/test.o
(lldb) target create "./bin/lldb-server"
Current executable set to '/home/david.spickett/build-llvm-aarch64/bin/lldb-server' (aarch64).
(lldb) settings set -- target.run-args  "g" "localhost:1234" "/tmp/test.o"
(lldb) b NativeRegisterContextLinux::DetermineArchitecture
Breakpoint 1: where = lldb-server`lldb_private::process_linux::NativeRegisterContextLinux::DetermineArchitecture(unsigned long), address = 0x00000000004e6644
(lldb) c
error: Command requires a current process.
(lldb) run
Process 3701235 launched: '/home/david.spickett/build-llvm-aarch64/bin/lldb-server' (aarch64)
Process 3701235 stopped and restarted: thread 1 received signal: SIGCHLD
Process 3701235 stopped
* thread #1, name = 'lldb-server', stop reason = breakpoint 1.1
    frame #0: 0x00000000004e6644 lldb-server`lldb_private::process_linux::NativeRegisterContextLinux::DetermineArchitecture(unsigned long)
lldb-server`lldb_private::process_linux::NativeRegisterContextLinux::DetermineArchitecture:
->  0x4e6644 <+0>:  stp    x29, x30, [sp, #-0x20]!

The clue for this one is “Native” as in “running on the target machine”.

If you definitely want InvokeCallback you would have to debug the lldb instead so:

$ ./bin/lldb-server g localhost:1234 /tmp/test.o
Launched '/tmp/test.o' as process 3701357...
lldb-server-local_build
Connection established.

$ /home/david.spickett/build-llvm-aarch64/bin/lldb -- /home/david.spickett/build-llvm-aarch64/bin/lldb /tmp/test.o
(lldb) target create "/home/david.spickett/build-llvm-aarch64/bin/lldb"
Current executable set to '/home/david.spickett/build-llvm-aarch64/bin/lldb' (aarch64).
(lldb) settings set -- target.run-args  "/tmp/test.o"
(lldb) c
error: Command requires a current process.
(lldb) run
Process 3701377 launched: '/home/david.spickett/build-llvm-aarch64/bin/lldb' (aarch64)
<...>
(lldb) target create "/tmp/test.o"
Current executable set to '/tmp/test.o' (aarch64).
(lldb) gdb-remote 1234
(lldb) Process 3701357 stopped
* thread #1, name = 'test.o', stop reason = signal SIGSTOP
    frame #0: 0x0000fffff7fcd100 ld-2.31.so`_start
<...>
Process 3701377 stopped
* thread #1, name = 'lldb', stop reason = signal SIGSTOP
    frame #0: 0x0000ffffee50cbf0 libc.so.6`__GI___libc_read at read.c:26:10
(lldb) b InvokeCallback
Breakpoint 1: 5 locations.
(lldb) c
Process 3701377 resuming
<set a breakpoint with a callback and run>

It’s at this point I advise using another debugger like gdb, to debug the lldb, just so you have different prompts to tell you which level you are at. (you can also make a settings file for one of the lldb to change what the prompt says, look at apropos prompt)

Also, when you go down from the lldb debugging the lldb, into the debugee lldb, the prompt won’t print again and you might think it’s stalled. It hasn’t. Just type some rubbish and hit enter, you’ll get a fresh prompt (crude, but it means you don’t accidentally run a real command).

Hope that helps.

As to learning what goes on in lldb-server vs lldb, this is a combination of signals really. A bit of source code layout, a lot of search and see where it’s called, and anything “native” is 99% sure in lldb-server.

One thing that would help you learn is to look at the gdb remote protocol and in particular lldb/source/Plugins/Process/gdb-remote/ProcessGDBRemote.cpp. This runs on the lldb side and talks to lldb-server.

Essentially, anything higher level than the questions you see in those functions, is probably going to happen in lldb. Aside from a few things like memory regions or shared library locations, which are actually probed server side.

1 Like
3

Thanks very much for your detailed reply，it’s really helpful for me.

1 Like