lldb problems on linux

Hi,

I recently build lldb from trunk (revision 187708) and source-level debugging isn’t working for me. It seems its not loading any source information. What is the best way to troubleshoot this?

Also make check-lldb doesn’t work on Linux when building with CMake. I just get error:

This script requires lldb.py to be in either /home/user/tools/llvm_3.4~svn187708/tools/lldb/build/Debug/LLDB.framework/Resources/Python, /home/user/tools/llvm_3.4~svn187708/tools/lldb/build/Release/LLDB.framework/Resources/Python, or /home/user/tools/llvm_3.4~svn187708/tools/lldb/build/BuildAndIntegration/LLDB.framework/Resources/Python

I get the same error with lldb-3.3.

Thanks!

Hi Kal,

For the second problem, you need to set the PYTHONPATH environment variable. Try this:

export PYTHONPATH=$llvm/build/Debug+Asserts/bin/lldb -P

Regarding the source information, I would start by using the following command within lldb (after you have created the target you want to debug):

target modules dump sections

If you don’t see debug sections in that list, then that’s the problem. If you do, try enabling DWARF logging (‘log enable dwarf all’) and see if anything obvious turns up in the output when you try to set a breakpoint.

-Andy

Hi Andy,
I tried
export PYTHONPATH=$llvm/build/Debug+Asserts/bin/lldb -P
but it didn’t work for me. If I run build/bin/lldb -P it outputs “build/lib7/site-packages” which doesn’t exist.
There is a directory build/lib/python-2.7/site-packages but if I set the PYTHONPATH to this directory I get the same errors.

I can import lldb; in python successfully though.

Any other ideas?

-Kal

Hmm… I’ve never seen the -P option print the wrong path. Looking at the code (in Host::GetLLDBPath) it doesn’t even look possible for it to print what you’re seeing.

On the other hand, the second directory you mention should be the correct one. If you set PYTHONPATH to that does “python -c ‘import lldb’” work?

-Andy

Hi Andy,
So I figured out the python issue. Host::GetLLDBPath() is broken. It was
failing for me because I am building in Release mode. It only works in
Debug mode by luck :slight_smile: The problem is lines 1035+ in
source/Host/common/Host.cpp. llvm::Twine should only be used for
temporary objects! See http://llvm.org/docs/ProgrammersManual.html#dss-twine

I have attached a patch this fixes the issue. I haven't found time to
investigate my other issue yet.

Thanks!
-Kal

lldb-get-python-path-fix.diff (1.57 KB)

Thank you Kal for the fix! Much appreciated :slight_smile:

I committed it in r187818.

So, just to clarify, you're still unable to run the test suite after the fix? Which distro are you on?

Thanks,
Dan

Hi Dan,
Sorry I wasn't clear. My fix fixes the test suite issue. The only
remaining issue is the source debugging issue. I haven't got to look
into that yet. I am on Debian Wheezy.
-Kal

I am still seeing issues with source-level debugging. "target modules
dump sections" has symbol entries. Also source level debugging is
working in gdb, so I know that symbols are available.
The strange thing is, I tried the same thing with a simple "Hello world"
program and source level debugging worked. Both programs are being
compiled with clang-3.4.

Can someone give me a tip where I can should put breakpoints in LLDB to
debug this?

Thanks,
-Kal

Enabling logging is probably a good first place to start. If you're not seeing source information, then maybe lldb is unable to parse some stuff that clang is generating… to see what the DWARF parser is doing, you can:

(lldb) log enable dwarf all

There's many logging categories that can be enabled; some are quite verbose..to see all of them, do:

(lldb) log list

Also, with the '-f' option, you can dump the output to a file for easier perusal.

Cheers,
Dan

I agree that logging is the best place to start. Without a better idea of what’s failing, it’s hard to say where to start.

You could try something like setting a breakpoint in the CommandObjectBreakpointSet::DoExecute() function, and stepping through from there. I think that’s likely to get you to the beginning of the DWARF processing if setting a breakpoint is the first thing you do after creating the target (or specifying it on the command line). It’s a long and winding road from there to the meaty bits, but without some hints from the log output it’s hard to say where a wrong turn might have been taken.

SymbolFileDWARF::ParseCompileUnit() is another potentially interesting starting place, but I think there’s a good chance you won’t get there if you aren’t seeing any source information.

-Andy

I enabled logging… I am seeing this when I run
(lldb) b main.cc:14

<… Lots of other info …>
main-thread Address Line Column File ISA Flags
main-thread ------------------ ------ ------ ------ — -------------
main-thread 0x00000000004afa70 8 0 1 0 is_stmt
main-thread 0x00000000004afd2e 14 0 1 0 is_stmt prologue_end
main-thread 0x00000000004afd69 16 0 1 0 is_stmt
main-thread 0x00000000004afd94 20 0 1 0 is_stmt
main-thread 0x00000000004afda3 23 0 1 0 is_stmt
main-thread 0x00000000004afe6c 24 0 1 0 is_stmt
main-thread 0x00000000004b017b 24 0 1 0 is_stmt end_sequence
main-thread 0x00000000004b0180 2292 0 2 0 is_stmt
main-thread 0x00000000004b02b3 2292 0 2 0 is_stmt prologue_end
main-thread 0x00000000004b0322 2292 0 2 0 is_stmt end_sequence
main-thread 0x00000000004b0330 494 0 78 0 is_stmt
main-thread 0x00000000004b0409 494 0 78 0 is_stmt prologue_end
main-thread 0x00000000004b04a1 494 0 78 0 is_stmt end_sequence
main-thread 0x00000000004b04b0 2292 0 2 0 is_stmt
main-thread 0x00000000004b05e3 2292 0 2 0 is_stmt prologue_end
main-thmain-thread DWARFDebugInfo::GetCompileUnitAranges() for “/home/kal/test/test” from .debug_aranges
main-thread DWARFDebugAranges::Sort(minimize = 1) with 23 entriesread 0x00000000004b0671 2292 0 2 0 is_stmt end_sequence

main-thread DWARFDebugAranges::Sort() 11 entries after minimizing (12 entries combined for 192 bytes saved)
main-thread 0x00000000: [0x45d670 - 0x45d7a4)
main-thread 0x00021392: [0x45d7b0 - 0x45d7be)
main-thread 0x00000000: [0x45dbf0 - 0x465c0c)
main-thread 0x0001d8d4: [0x465c10 - 0x467120)
main-thread 0x00021392: [0x467120 - 0x468172)
main-thread 0x0002429a: [0x468180 - 0x485d9c)
main-thread 0x0005cd35: [0x485da0 - 0x485e1f)
main-thread 0x0005dd15: [0x485e20 - 0x4865e5)
main-thread 0x0005fff2: [0x4865f0 - 0x486b7c)
main-thread 0x00061b4e: [0x486b80 - 0x487eff)
main-thread 0x00064a9d: [0x487f00 - 0x488295)
Breakpoint 1: no locations (pending).
WARNING: Unable to resolve breakpoint to any actual locations.

The address 0x00000000004afd2e for line 14 is correct. If i run “b main” I also get there. But that address isn’t in the Aranges list.

-Kal

If I apply the following to ignore .debug_aranges source debugging works again:

Index: source/Plugins/SymbolFile/DWARF/DWARFDebugInfo.cpp

Hmm, what compiler/version are you using to build the inferior you are trying to debug? Any chance you can file a bug at llvm.org/bugs and attach the problematic binary?

Thanks,
Dan

I am not able to post the source/binary of this inferior. I built it with clang-3.4. If I build with g++4.7 lldb loads the source information correctly, so this may be a clang bug if you think the generated dwarf info is wrong. On the other hand gdb does handle this binary without a problem.

-Kal

I figured out how to reproduce this. Just compile with clang -fsanitize=address any program and you should see what I am seeing.

-Kal

It sounds like the .debug_aranges section the compiler is generating is not correct, and that is what is messing everything up. If we have a .debug_aranges section, we use it. If it is missing, then we auto generate it by pulling in all DWARF and calculating it by hand. This is bad because it consumes time and memory to pull in all DWARF just to generate the address ranges.

File a bug against clang with your repro steps.

Greg

Is there really a requirement that .debug_aranges must cover every debug section in the entire program? I couldn’t find this in the DWARF spec. Seems like LLDB should try .debug_aranges first then fallback gracefully. The way it is currently, if you link in any object file that happens to have a .debug_aranges section, then you can’t debug the rest of the program with LLDB. The test below demonstrates the problem. Note that gcc generates a .debug_aranges section and clang does not.

What good is an accelerator table that doesn't do its job? Should we do this for all things, like lookups by name? By address? Seems a little lame to emit accelerator tables and not use them, or use them and fall back to very very expensive tables that you must manually generate. Debugging very large C++ apps will perform VERY poorly if we don't use the accelerator tables and have to parse ALL DWARF for all translation units just to get an address map.

Although clang doesn't generate the .debug_aranges, it could and it should. Also, on MacOSX, dsymutil, the DWARF linker, will create proper .debug_aranges section when a final DWARF file is generated from all the .o files.

Greg

Just to follow up on this thread, the following two commits should fix this:

[lldb] r190365 - Fixed parsing of the .debug_aranges section so that LLDB will still work correctly even if arange data is not available for every object file in the program.

[llvm] r191052 - Added support for generate DWARF .debug_aranges sections automatically.

With these applied, lldb will function with only partial arange data, and clang will always generate arange data. Either patch should fix the problem here.

Richard Mitton
richard@codersnotes.com