Let's add some documentation about how to debug lldb itself

fwiw I personally prefer to run the debugee-lldb in one window, and then attach to it from lldb in a separate window, so the two lldb’s aren’t fighting for console input and I don’t have to change the prompts to disambiguate which one I’m typing in to.

See this is exactly the sort of thing I was thinking about, I never thought to do this.

I just type some garbage into the debugee lldb and the “unknown command” error will print a new prompt. A lot less elegant.

I guess the one case where I often run lldb on lldb is when I’m looking to catch a crash, and not interacting with the top lldb intentionally before that. I still have the problem if I have top-lldb > lldb > a.out and I press control-c, top-lldb will see it first and give me the top-lldb prompt.

Cool initiative!

I haven’t touched lldb code in quite some time, but I think I remember some of the tests were written in python. Maybe a few words on debugging those would be helpful.

One thing that I’ve been struggling with lately - how do I debug lldb using lldb-vscode, and type in commands? I can blast in commands with -o in the args property in launch.json, but once I’m running I can’t figure out how to interactively type in commands.

In general, I think our vscode solution could use more documentation and attention. All we’ve really got is @vadimcn 's CodeLLDB on the marketplace. It’s a good solution, but we should do more with lldb-vscode.

Most of our tests are written in Python actually. We have a little snippet on the website but I agree there’s room for improvement!

Do folks run their just-built lldb from the build dir? A few years ago I considered having CMake generate a .lldbinit in the build root with to change the inferior LLDB’s prompt.

+1, especially when running under tmux, this can avoid editline getting confused.

Great initiative!

If possible, I’d be happy to contribute to the documentation, even though I’m a beginner in the subject.

gdb does/did that same thing. The main drawback is that lldb doesn’t read .lldbinit in current working directory by default, for security reasons.

Cool, didn’t know there was prior art. The “security” feature is the reason I didn’t pursue it further. But if folks think that’s useful, we could special case the build dir in a debug/assert build or with a CMake variable.

I’m a big fan of this. I would also like to contribute to this cause with what I can.

In addition to the list you’ve provided, some other things I think would be useful:

  • General lldb tips
    • How to use some more advanced breakpoint features (e.g. conditional breakpoints)
    • How to enable lldb logging and how to redirect it to a file for later analysis
      • Somewhat related: It might be useful to create an “intro to gdb-remote protocol”. We have a nice list of each packet and what it is supposed to do, but no intro to the subject itself.
    • How to manipulate the environment variables of the inferior (I’ve seen some confusion from end users on this actually)
    • How to use apropos to find commands or settings that are possibly relevant
  • A more complete introduction to lldb’s python integration
    • How to use it, how to write new things, etc. What we have today is a useful introduction, but I tried to use it the other day and found it rather terse.
    • How our swig binding code is laid out (I’ve had to learn this the hard way a few times, documentation would have been nice).
  • Strategies for alternate ways of accomplishing common tasks. For example, if you try to evaluate an expression but it fails, what are some alternate ways you could attempt to answer the question that the expression evaluation would have answered?

I’ve been thinking about improving documentation for a while, I’d also like to contribute these semi-related ideas:

  • A guide on automating lldb workflows. More specifically, how to combine the lldb command line flags and python integration to automatically perform certain tasks on things like corefiles and bad binaries.
  • A more beginner-friendly way to tell users about certain aspects of their debugging session. For example, how do I know if I lldb found my debug info? How do I perform source remappings? etc.

+1 on this. My experience debugging lldb with itself using 1 terminal is fraught with confusion and frustration. Attaching to an existing lldb instance in a separate terminal makes it extremely clear what is happening where.

Perhaps @clayborg or some of his colleagues from Meta can better help you with this. IIRC they are using lldb-vscode quite extensively.

Worth mentioning that there are some data formatters for lldb data types (in lldb/examples/summaries/lldb). We should also extend those and if possible set up some testing for them so they don’t bit-rot. The situation isn’t as bad as llvm/clang, lldb has few data structures that you can’t just look at. But OTOH, having a Module * show its name w/o having to drill down into the structure is such a time saver.

This are all good ideas, sounds like we have got 2 things:

  • How to debug lldb using lldb.
  • How to make the best use of lldb, regardless of debugee (but of course including lldb itself).

Between GDB’s documentation and our scattered text files the information is there but I agree it could be put together.

Also it would be a good place to state (/restate, it’s probably somewhere on the website) our goals with regards to compatibility with combinations of lldb/gdb and lldb-server/gdbserver and other “compatible” stubs.

We have had recent questions that would have been helped by such material e.g. Traversing member types of a type.

1 Like

It would be nice if LLDB had more material towards formatter writers. For instance, today I’m using script print(...) to debug various expressions I’d like to put into a formatter. I think it would be easier to have a Python debugger inside formatters if my whole LLDB session was Python-scripted. I tried, but resorted to print-style debugging in the end.

One from me is how to reduce crashes involving lldb and Linux ptrace down into small examples that don’t need lldb. For example when you find a kernel crash.

Which I know how to do now, but it’s not obvious, and is mostly thanks to the MVP of debugger articles How debuggers work: Part 1 - Basics - Eli Bendersky's website.

1 Like

lldb’s instance of the Python interpreter is a pretty vanilla interpreter, so it does all the things a normal Python interpreter does, including if you put breakpoint() in the code of a Python formatter, it will break into the python debugger when the formatter is executing, and you can use all the pdb commands to step through the python, inspect variables, etc.

That’s obvious in hindsight, but a little magical and it’s not obvious you would expect it to work. So we should indeed include that in the documentation. This doesn’t just work for formatters, but for all the python affordances in lldb.

1 Like

I think another good documentation add would be how to use pdb to debug API tests. I generally resort to adding in print() or self.runCmd() calls do get data out, but that can become very cumbersome.

Indeed, though again, the message is "anywhere that lldb is running python code, pdb (or really whatever python debugger you have installed) should interrupt that python code execution and give you control. Putting breakpoint() anywhere in the test file will stop execution.

I use this all the time, not just for python debugging. For instance, it’s often the case that I want to debug lldb when it’s performing a particular line in the test file, but there’s lots of code before that that is also going to hit the breakpoints I need to debug the line I want to. So I just put breakpoint() in the test before that line, run:

lldb-dotest -p <TestFile.py> -f <test_case> -d

Then attach with lldb, disable all my breakpoints, and continue. Then when I hit the python breakpoint in the test, I reenable all my breakpoints in lldb, run continue at the pdb prompt and I’m all set to debug the line I wanted to…


On Aug 3, 2023, at 6:47 AM, Ted Woodward via LLVM Discussion Forums notifications@llvm.discoursemail.com wrote:

August 3

I think another good documentation add would be how to use pdb to debug API tests. I generally resort to adding in print() or self.runCmd() calls do get data out, but that can become very cumbersome.

Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

[lldb][Docs] Add page about debugging lldb itself by DavidSpickett · Pull Request #65332 · llvm/llvm-project · GitHub for the initial lldb and lldb-server parts that were in my initial message.

Anything to do with debugserver I have no idea, so if someone wants to add that on top that would be great. I’ve just said the architecture is mostly the same.

I’ll go through the rest of the answers/requests and what I’m familiar with I’ll put up patches for, what I’m not, I’ll politely request an expert to document it for me :slight_smile:

Our testing page already had some debugging content so I’ve made the debugging page link to that ([lldb][Docs] Link to testing page from debugging page · llvm/llvm-project@3ae76e4 · GitHub).

We have some places you can’t use Pdb (AttributeError when using Pdb in lldb's script interpreter · Issue #66356 · llvm/llvm-project · GitHub) still.

Using Pdb is basically all I do to debug Python so if others have ideas please add them to the testing page, and to the debugging page for anything else.

1 Like

There are plenty of ideas in the thread too if anyone wants to volunteer their time (I need to prioritise other things right now). Happy to review any and all of it.

1 Like