Symbolicate user processes when kernel debugging

I'm using lldb to debug the OS X kernel, and it works great.
I would like to have more flexibility in analysing user programs while
debugging the kernel itself,
and specifically symbolicate the code of the user programs.

For example I often use the command showthreaduserstack defined here
http://opensource.apple.com//source/xnu/xnu-2422.1.72/tools/lldbmacros/userspace.py
to take
a look at the user stack of a process running in kernel mode that just
scripts the process of
obtaining the thread saved state, but the output unfortunately isn't
symbolicated.

Is there a way to add symbols for a user process (programs and shared libs?)
I looked into the target modules add command, but when I try to add a
copy of the executable
it just says that the file I pick doesn't exist (even though it clearly exist).
Also I'm not entirely sure how that would work since the user space
addressing space changes
for every process, even if I manually set the loading address.
Would that work only for that specific process and execution?

Regards,
John

The right way to do this is to say "I want a GDB server port for user space process 123". The python would then start up a socket that can be connected to that can vend the information about the user space process directly through a dedicated GDB server port. Memory reads would translate the memory asked for through the GDB server port into a physical address and do the read for you as if the memory read came from user space process 123. I know someone had this code working here at Apple, but I am not sure if it made it into the macros. You might check around for such a thing as it might already be in there. Then you can also read memory and read registers just as you would with a core file. Then you can skip all of the manual symbolication stuff as the process will set itself up correctly if the GDB server is responding to all the right questions.

So check around and make sure this isn't already checked into the code.

Greg Clayton

"I want a GDB server port for user space process 123"

How would I start this gdb server? Do you mean a gdb-server running in
the target userspace? Wouldn't that make impossible to use it when the
kernel is stopped?

I tried searching around and the only resources I found is this old
macros file for gdb
(http://opensource.apple.com/source/xnu/xnu-1456.1.26/kgmacros) that
had a switchtouserthread command that seems to do something similar to
what I want to achieve. (but obviously I want to use lldb so that
doesn't apply, also I'm not sure it would work since it is pretty
old).
The other interesting file I found is
https://opensource.apple.com/source/xnu/xnu-3247.1.106/tools/lldbmacros/usertaskgdbserver.py?txt
that has a beginusertaskdebugging that from the description seems to
do what you were describing, but strangely it doesn't seem to be
available/implemented?

John Otter

"I want a GDB server port for user space process 123"

How would I start this gdb server? Do you mean a gdb-server running in
the target userspace? Wouldn't that make impossible to use it when the
kernel is stopped?

No, the python script that is debugging the kernel would make a socket on a new port and hand that out. It is very easy to write a GDB server in python. Each connection would know what user space process it is answering questions for. For example we say "create GDB server for process 123" and we create a GDBServerPython class and give it user space process 123. When the connection asks about the threads in the process, it would read the threads from the kernel data structures and return the info on the threads for user space process 123. If memory is requested like "memory read 0x1000 --count 256", you would look through the TLB entries in the kernel, map "0x1000" to a physical address, and return the correct bytes for that memory read. When registers for a given thread are requested, you would find the thread data structure in kernel memory for the thread and return the correct registers. That is enough to fake a user space connection to a user space process.

I tried searching around and the only resources I found is this old
macros file for gdb
(http://opensource.apple.com/source/xnu/xnu-1456.1.26/kgmacros) that
had a switchtouserthread command that seems to do something similar to
what I want to achieve. (but obviously I want to use lldb so that
doesn't apply, also I'm not sure it would work since it is pretty
old).
The other interesting file I found is
https://opensource.apple.com/source/xnu/xnu-3247.1.106/tools/lldbmacros/usertaskgdbserver.py?txt
that has a beginusertaskdebugging that from the description seems to
do what you were describing, but strangely it doesn't seem to be
available/implemented?

I looked into it and we have a solution that does this internally at Apple, but it isn't in the public XNU LLDB macros. I will check if they are willing to give out any details on this so others could use it.

Greg clayton