TL;DR Why is it that if I use clang in a two step process to assemble and link I get debug info. But if I do it in one step directly from source, I don’t?
I asked this on the discord #clang section, and on stackoverflow, but then thought that it might really be an lldb question and was directed to this platform.
If you add -### to a clang command line, the driver will dump out the command lines it’s using to run the actual tools. Comparing the two scenarios could be informative.
Thanks for the quick response and the -### tip. Ya know, I did add -v to the command line and compared the results, but didn’t see anything obvious. I’ll take another, more thorough look though. What would you expect I’ll find?
Oh, -v probably shows you the same thing. I suggest that first, to see whether the driver dropped something, mainly because it’s quick and easy not because it’s likely. Make sure you see -debug-info-kind=limited in the assembler command (the one where the first option on the line is -cc1as).
Next after that would be to poke around in the compiled binaries, looking at the object-file sections to see whether the debug info is there at all. You’re on Mac, right? so they’re MachO not ELF. Try llvm-objdump --headers <your-file-here> and see what comes up. Off the top of my head I don’t remember the MachO section names for DWARF.
Okay, I thought I had it figured out, but then not so much. BTW @pogo59 the assembler command lists -debug-info-kind=constructor not -debug-info-kind=limited. Important?
I ran clang -g -o factorial factorial.s -v to get the steps, then I copied and pasted them one by one into a terminal. The final command runs dsymutil. I did not execute that, and I had my symbols. I then executed it, got the .dSYM bundle that it generates, and lost my symbols (ironically). I then deleted that bundle, and saw that my symbols were back. Thought I had a remedy. (e.g. delete the bundle after building). So I ran clang -g -o factorial factorial.s and rm -rf factorial.dSYM and fired up lldb. No joy.
Thinking the temporary files may be an issue, I tried clang -g -o factorial factorial.s -### to get the commands without executing them. Then I executed them one by one copy/paste leaving out the last step, calling dsymutil. Success. Ran dsymutil. Broken. Ran rm -rf factorial.dSYM. Success.
BTW, I’m testing for success by launching lldb and executing source info -f factorial.s which I expect will show me something like:
To summarize, if I execute the steps from -### myself, by hand, and delete the .dSYM bundle, it works. If I run the clang -g -o factorial factorial.s and delete the .dSYM bundle, it doesn’t work.
I must be so very close though, because this makes no sense whatsoever.
It occurs to me that it is entirely possible that I’m mis-using some terminology here. By losing my symbols, I mean that when I set a break point, lldb can’t show me the source. When it is working, I expect to see my source when I break on main (for example) and the break point is hit:
chris@goldfish chapter11 % !lldb
lldb factorial
(lldb) target create "factorial"
Current executable set to '/Users/chris/Dev/assembly/learning-assembly/chapter11/factorial' (x86_64).
(lldb) b main
Breakpoint 1: where = factorial`main + 2, address = 0x0000000100003f8b
(lldb) r
Process 35823 launched: '/Users/chris/Dev/assembly/learning-assembly/chapter11/factorial' (x86_64)
Process 35823 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x0000000100003f8b factorial`main at factorial.s:13
10 pushq $0
11
12 # push NUMBER ... 1 onto the stack
-> 13 movq $NUMBER, %rcx
14 pushvalues:
15 pushq %rcx
16 loop pushvalues
Target 0: (factorial) stopped.
warning: This version of LLDB has no plugin for the language "assembler". Inspection of frame variables will be limited.
(lldb)
What I’m seeing when “it” doesn’t work — what I’m describing as losing my symbols in a one-step build is this:
chris@goldfish chapter11 % !ll
lldb factorial
(lldb) target create "factorial"
Current executable set to '/Users/chris/Dev/assembly/learning-assembly/chapter11/factorial' (x86_64).
(lldb) b main
Breakpoint 1: where = factorial`main, address = 0x0000000100003f89
(lldb) r
Process 35929 launched: '/Users/chris/Dev/assembly/learning-assembly/chapter11/factorial' (x86_64)
Process 35929 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x0000000100003f89 factorial`main
factorial`main:
-> 0x100003f89 <+0>: pushq $0x0
0x100003f8b <+2>: movq $0x4, %rcx
factorial`pushvalues:
0x100003f92 <+0>: pushq %rcx
0x100003f93 <+1>: loop 0x100003f92 ; <+0>
Target 0: (factorial) stopped.
(lldb)
On Darwin systems, the linker does not ever insert DWARF debug information into a binary. Instead it uses one of two modes:
Debug in .o file: leave the DWARF in the .o files and write a “debug map” into the binary that tells lldb where the .o files are and how the linker mapped symbols from .o to binary
Create a dSYM (using dsymutil) which does the same linking job lldb does and produces a stand alone debug file.
The problem with the “one step” build process is that when you don’t specify a .o output, clang creates a temporary one that it deletes when the linking is done. That rules out method 1 above, so clang makes a dSYM in that case before deleting the temporary .o file. TTTT, I’ve never ensured this happens the same way with .s file but it would be surprising if it didn’t.
The dSYM should be equivalent to the .o files & debug map, if that’s not true that’s a bug in dsymutil (why Adrian was asking for info about the dSYM…)
It’s also curious that anything works on deleting the dSYM, since that implies that the .o files are still hanging around somewhere, which is unexpected.
You can view the debug map (which we cannily reused STABS entries for) by doing:
$ nm -ap <path_to_binary> | grep " OSO"
that will list all the debug files the debug map was told about. If you look at the N_FUN entries you’ll also see all the functions we were told about, etc…
You can see whether lldb read in any .o files for debugging purposes by issuing:
(lldb) image list -g
All the .o files will be at the bottom. There’s a bug in this listing (just in the listing, not the search) - we don’t list .o files we found in .a files, but I don’t think that’s relevant in your case.
Wow! I think this explains everything. Thanks for the clear and complete explanation.
It is not working exactly the same in this case. Seems there is a little daylight at the top.
That appears to be a side effect of my executing by copy/paste. They are hanging around because nobody (me) deleted them. In the “one step” build, they are sensibly removed after the build is done. But I just left them under /tmp…
Looks like the dSYM bundle info is preferred over the .o files. I used a “two step” build and then ran dsymutil, and this reverted me to the broken behavior. The factorial.o file was not in the image list. Deleting the bundle then showed my factorial.o (from the two-step build) in the image list.
Makes me wonder a little bit what’s missing from the dSYM bundle that is available in the .o file. Thanks! TIL…
The dwarfdump output fromt he .dSYM and the .o isn’t fundamentally different, so I wonder if there’s a bug in LLDB’s handling of DW_TAG_label that only exists in the dSYM SymbolFile plugin?