How to debug to learn LLVM Backend

Hello Experts,

I am very new to LLVM and am trying to understand LLVM Backend.

I have a question related to an older thread (https://discourse.llvm.org/t/could-you-give-me-some-suggestions-as-to-learning-llvm/52646/2) which I hope someone can answer please.

In the reply to the question in the above thread, it is mentioned that a very good way to lean is to: “Study a backend by stepping its source code with a debugger,”

Can someone please elaborate more on this? What kind of debugger is suggested for this? Can someone maybe give a small example of how one would resolve a roadblock in learning process by using a debugger to understand the solution? (I realize that the answer to my question might be very clear to experts here, but unfortunately as a beginner lots of things are not very clear to me :< )

Thank you very much in advance
UHD

Stepping through an entire compilation can be equal parts tedious and overwhelming, so don’t be too discouraged if you find that to be the case. The more you can narrow down the portion of the code you want to follow, the more useful it is, but of course, if you knew what portion to look at already, you wouldn’t be asking the question :slight_smile:

Anyway, personal opinions aside -

You’ll need a build of llvm in debug mode, which can be done by passing -DCMAKE_BUILD_TYPE=Debug to cmake when configuring the build.

Building in debug mode will take more time, more disk space and more RAM, and there are ways to handle that causes a problem on your machine.

Once you have that you’ll need a debugger. The two obvious ones are gdb and lldb. If you are on Linux you likely have gdb already, and you can install an lldb from apt if you want to use that. As a beginner to debuggers, I would use gdb (despite being an lldb contributor myself). There’s just more material out there covering it.

(other debuggers are available but I don’t know the details, for example, for Microsoft Visual Studio)

Next you need to figure out what program the debugger should run. Take the clang/llc/opt/whatever command you want to step through and use that:

gdb -args <the command>
or
lldb <the program> -- <arguments to the program>

The command might be llc myfile.ir or clang test.c -fsome-option. Note that clang may spawn a new process for the “driver” aka clang -cc1, so if you are debugging a clang command, sometimes you will have to add -### to the command to get the cc1 command and debug that instead.

If you want to step all the way from the beginning, set a breakpoint on main. b main works in both debuggers.

If you want to know how the program got to a location in source code, you can place a breakpoint there directly. For example, where a certain string is printed, or a variable is assigned to some value (which you find by searching the source code).

b somefile.cpp:1234 or b someFunctionName

Now you tell the debugger to run the program. run works for either debugger.

Eventually it’ll hit a breakpoint. From here you can “step” in various ways through the program. Continuing and Stepping (Debugging with GDB) is a good reference for gdb and lldb has equivalents.

The main differences are whether you want to step “over” a function call, or into it. For example:

int i = 1; // you are here
foo(i);
i = 2;

You can choose to step over the call to foo or into the foo function and see what it does. If you’re really stepping the entire thing it’ll be a lot of step in, but often step over is what you want.

There is also finish to step until you leave the current function. Useful if you stepped in and it turns out to be some boring helper function.

Now if you’ve placed a breakpoint in a specific place, you can use the “back trace” to see how you got there. This is the bt command in both debuggers iirc.

This will show you a list of function calls that are how you got to the current location. Using the up and down commands you can move through that list and see what was passed into the functions.

So for example, if you know where an error message is printed, you can breakpoint there and then look up the callstack to see how you got there. The location might be buried in a million lines of source code, but if you know that somehow you got there, the debugger can decode the path that was taken.

Note though that this is what you might call the “program flow” as in what branches were taken. Sometimes the “data flow” is more interesting, which is what data was set in one place and later used in another. Debuggers aren’t so great at that side of things, but there are “watchpoints” that are like code breakpoints but for data. I won’t go into those here but feel free to ask about them if you do look into it.

An example, say I want to know how clang --version works. I know it prints Thread model:.

david.spickett@tcwg-jade-03-dev:~/build-llvm-aarch64$ ./bin/clang --version
clang version 19.0.0git (https://github.com/llvm/llvm-project.git 5f50abb6d5f36a35b206b673f10a88635ba54a43)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/david.spickett/build-llvm-aarch64/bin
Build config: +unoptimized, +assertions

So I guess that that literal string is somewhere in the source code. Find in files finds it in clang/lib/Driver/Driver.cpp in a function Driver::PrintVersion.

Lucky for me, this happens in the early part of clang before it might fork a process, so I don’t need to use -### here.

So let’s run that under the debugger:

$ ./bin/lldb ./bin/clang -- --version
(lldb) target create "./bin/clang"
Current executable set to '/home/david.spickett/build-llvm-aarch64/bin/clang' (aarch64).
(lldb) settings set -- target.run-args  "--version"

Break in that function:

(lldb) b Driver::PrintVersion
Breakpoint 1: where = clang`clang::driver::Driver::PrintVersion(clang::driver::Compilation const&, llvm::raw_ostream&) const + 36 at Driver.cpp:2009:7, address = 0x000000000667545c

Run the program:

(lldb) run
Process 3224028 launched: '/home/david.spickett/build-llvm-aarch64/bin/clang' (aarch64)
Process 3224028 stopped
* thread #1, name = 'clang', stop reason = breakpoint 1.1
    frame #0: 0x0000aaaab111f45c clang`clang::driver::Driver::PrintVersion(this=0x0000ffffffffdae0, C=0x0000aaaabf0b4c10, OS=0x0000aaaabefff008) const at Driver.cpp:2009:7
   2006 }
   2007
   2008 void Driver::PrintVersion(const Compilation &C, raw_ostream &OS) const {
-> 2009   if (IsFlangMode()) {
   2010     OS << getClangToolFullVersion("flang-new") << '\n';
   2011   } else {
   2012     // FIXME: The following handlers should use a callback mechanism, we don't

See how we got to this point:

(lldb) bt
* thread #1, name = 'clang', stop reason = breakpoint 1.1
  * frame #0: 0x0000aaaab111f45c clang`clang::driver::Driver::PrintVersion(this=0x0000ffffffffdae0, C=0x0000aaaabf0b4c10, OS=0x0000aaaabefff008) const at Driver.cpp:2009:7
    frame #1: 0x0000aaaab1119908 clang`clang::driver::Driver::HandleImmediateArgs(this=0x0000ffffffffdae0, C=0x0000aaaabf0b4c10) at Driver.cpp:2167:5
    frame #2: 0x0000aaaab1119404 clang`clang::driver::Driver::BuildCompilation(this=0x0000ffffffffdae0, ArgList=ArrayRef<const char *> @ 0x0000ffffffffd5e8) at Driver.cpp:1528:8
    frame #3: 0x0000aaaaac6f0114 clang`clang_main(Argc=2, Argv=0x0000fffffffff2e8, ToolContext=0x0000fffffffff0c0) at driver.cpp:361:44
    frame #4: 0x0000aaaaac7249cc clang`main(argc=2, argv=0x0000fffffffff2e8) at clang-driver.cpp:17:10
    frame #5: 0x0000fffff7b37e10 libc.so.6`__libc_start_main(main=(clang`main at clang-driver.cpp:15), argc=2, argv=0x0000fffffffff2e8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=<unavailable>) at libc-start.c:308:16
    frame #6: 0x0000aaaaac6ef428 clang`_start + 52

From there I might decide I’m more interested in one of the earlier functions and breakpoint there instead, then step from that point forward.

However if my command had 20 options instead of 1, I may end up hitting 19 breakpoints for options I don’t care about. The more specific you can be, the less you have to filer manually.

From a high level this workflow is:

  • Break where you know the program gets to, because of some output or decision it makes.
  • Look back up the callstack to see where the critical decision point might be.
  • Break in that function instead and run again.
  • Repeat until you find the decision point.

There exist “reverse debugging” using tools like GitHub - rr-debugger/rr: Record and Replay Framework that allow you to do this without having to reset each time, but the logic is the same overall.

Hi DavidSpickett,

Thank you so much for such a detailed reply. It is a tremendous help for me to get me started.

And yes, as you mentioned, the “data flow” is another big part of this (and a mystery to me!) I.e. who is the consumer of this data that I am setting here?

So can you please talk about that?

Thank you again for your detailed reply.

Regards,
UHD

Hi @DavidSpickett,

Thank you so much for such a detailed reply. It is a tremendous help for me to get me started.

And yes, as you mentioned, the “data flow” is another big part of this (and a mystery to me!) I.e. who is the consumer of this data that I am setting here?

So can you please talk about that?

Thank you again for your detailed reply.

Regards,
UHD

:+1:

So I don’t know about tools specific to tracking data flow (I have heard good things about DDD but haven’t used it myself), but you can do some of it with a combination of source reading and watchpoints in a normal debugger.

To clarify first, generally a so called “break” point is a code breakpoint. If you try to execute the instruction at this address, halt the program and enter the debugger.

A “watch” point watches for loads from and/or stores to a location. Depending on the hardware capabilities you can say things like “when this address is written with a size of X and value Y, stop the program”. When the program stops you’ll get that same backtrace I showed before. Except, you could be stopped anywhere in the program now, anywhere that tried to access that location.

How do you find the location(s) to watch? You’ll have to read the sources, or use a code breakpoint to find out what’s being read at the key moment, then watch that location and re-run the program.

Sometimes there’s several redundant copying steps you’ll have to work back through. One place might write the key value to the location that is read to decide control flow, but that value may have been decided much earlier and moved around a few times.

gdb allows “software watchpoints” which mean it always stops whenever there is an access, then evaluates an expression you provide it. So for instance you could say the equivalent of “if the value is 1 and this->name is some_special_instruction” then stop.

Only if that expression is true will it enter the debugger, if it’s false it’ll just carry on and not bother you. This means you wouldn’t have to manually ignore 99.9% of hits to get the 0.01% you care about, if there are, for instance, thousands of backend instructions being processed.

If you have the ability to rebuild the program, you can also emulate watchpoints like this by writing extra code into the program. For example if you have a setter method:

class Foo {
public:
  void SetDoSpecialStuff(bool v) { do_special_stuff = v; }
private:
  bool m_do_special_stuff = false;
  const char* m_name;
};

You could change the setter to:

void SetDoSpecialStuff(bool v) {
  if (strcmp(m_name, "interesting_instruction_name") == 0)
    __builtin_trap();
  do_special_stuff = v;
}

And that effectively is a conditional watchpoint but it’ll run much faster than the software watchpoint gdb provides. __builtin_trap inserts what’s effectively a breakpoint but embedded into the program, so you’ll enter the debugger and be able to backtrace.

If you want to know who uses the data, do this to the getter instead.

This technique only works with something that has a well defined code path of course. If the value is set by a memcpy of a thousand structures from one page to another, this won’t help and a normal watchpoint is needed.

Sometimes catching one write isn’t enough. Perhaps you know who writes the value but it’s the order of the values that’s in question. Replace the trap with a printf and now you’ve got a log of the whole sequence.

If you can’t rebuild the binary, you can do this by attaching a custom command to a debugger breakpoint. In lldb terms something like expr interesting_value; continue.

The techniques you use will always be a mixture of personal preference and the circumstance of the current investigation. So don’t stress that you’re doing it wrong, if it found you the answers you wanted it was probably fine (even if inefficient).

Hello @DavidSpickett,

(I apologize for a long post :frowning: but I wanted to make sure that I provide all the details)

So l am trying to follow the method you described for program flow :slight_smile: to learn/understand a target’s source code. I am trying to investigate lets say Frame handling in AVR (I am choosing AVR because it seems to be much smaller target in terms of number of files as compared to Sparc/MIPS etc). I am doing the following:

The approach I am taking is that: I try to learn about source code of a target (Frame handling in AVR in this case) by creating a small C code file and run LLDB on the command that generates Assembly file and then (in the LLDB session) I put a break point on a line in target source code (“AVRFrameLowering.cpp” in this case) and investigate what were the functions called up to this point.

NOTE1: Actually I am not sure if my approach described above (i.e. use LLDB breakpoints on target’s source file while using command that generates target Assembly) is correct to begin with?? So please feel free to comment and let me know if there is a better way to learn a target’s source code

NOTE2: Please note that the steps I am doing below might not be the best way to do them. So please feel free to suggest me better options :slight_smile:

Step1 (Initial Setup):
I am using Linux for my development.

I download LLVM source and place it at:
/home/uhdesign/llvm_work/llvm_download/

I create a build directory at:
/home/uhdesign/llvm_work/llvm_build/

Then I build it using:
$> cd /home/uhdesign/llvm_work/llvm_build/
$> /home/uhdesign/llvm_work/cmake_build/bin/cmake -DCMAKE_C_COMPILER=/home/uhdesign/llvm_work/gcc_13_build/bin/gcc -DCMAKE_CXX_COMPILER=/home/uhdesign/llvm_work/gcc_13_build/bin/g++ -DLLVM_TARGETS_TO_BUILD="X86;Sparc;AVR" -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_PROJECTS="clang;lld" -DLLVM_ENABLE_ASSERTIONS="ON" -DCMAKE_INSTALL_PREFIX=/home/uhdesign/llvm_work/llvm_build /home/uhdesign/llvm_work/llvm_download/llvm-project/llvm

I install LLDB at standard location via command:
$> sudo apt install lldb

Step2 (Write a simple C program and generate AVR Assembly):
I write a simple C program as:

$> cat test_add.c
int gl_a;
int gl_b;

int my_func(int c, int d)
{
  int out;
  out = gl_a + gl_b + c + d;
  return out;
}

I place the above code at a new location:
/home/uhdesign/llvm_work/my_tests/

Then I generate Assembly for AVR by:

$> cd /home/uhdesign/llvm_work/my_tests/
$> /home/uhdesign/llvm_work/llvm_build/bin/clang --target=avr -S test_add.c

The above command throws this message on terminal (so I am not sure what I am missing):

$> clang: warning: no target microcontroller specified on command line, cannot link standard libraries, please pass -mmcu=<mcu name> [-Wavr-rtlib-linking-quirks]

But it (i.e. the above command) does generate an Assembly file test_add.s as below:

	.text
.set __tmp_reg__, 0
.set __zero_reg__, 1
.set __SREG__, 63
.set __SP_H__, 62
.set __SP_L__, 61
	.file	"test_add.c"
	.globl	my_func                         ; -- Begin function my_func
	.p2align	1
	.type	my_func,@function
my_func:                                ; @my_func
; %bb.0:                                ; %entry
	push	r28
	push	r29
	in	r28, 61
	in	r29, 62
	sbiw	r28, 6
	in	r0, 63
	cli
	out	62, r29
	out	63, r0
	out	61, r28
	std	Y+6, r25
	std	Y+5, r24
	std	Y+4, r23
	std	Y+3, r22
	lds	r24, gl_a
	lds	r25, gl_a+1
	lds	r18, gl_b
	lds	r19, gl_b+1
	add	r24, r18
	adc	r25, r19
	ldd	r18, Y+5
	ldd	r19, Y+6
	add	r24, r18
	adc	r25, r19
	ldd	r18, Y+3
	ldd	r19, Y+4
	add	r24, r18
	adc	r25, r19
	std	Y+2, r25
	std	Y+1, r24
	ldd	r24, Y+1
	ldd	r25, Y+2
	adiw	r28, 6
	in	r0, 63
	cli
	out	62, r29
	out	63, r0
	out	61, r28
	pop	r29
	pop	r28
	ret
.Lfunc_end0:
	.size	my_func, .Lfunc_end0-my_func
                                        ; -- End function
	; Declaring this symbol tells the CRT that it should
	;clear the zeroed data section on startup
	.globl	__do_clear_bss
	.type	gl_a,@object                    ; @gl_a
	.section	.bss,"aw",@nobits
	.globl	gl_a
gl_a:
	.short	0                               ; 0x0
	.size	gl_a, 2

	.type	gl_b,@object                    ; @gl_b
	.globl	gl_b
gl_b:
	.short	0                               ; 0x0
	.size	gl_b, 2

	.addrsig
	.addrsig_sym gl_a
	.addrsig_sym gl_b

Since I see that the assembly file is generated so I take that the command mentioned in this step to generate Assembly works.

Step3 (Run LLDB and set breakpoint):
Now I want to rerun the command from Step2 but this time I want to use LLDB and insert a breakpoint at Line 97 of AVRFrameLowering.cpp

I chose line 97 because it has the following code which I think could be useful to learn about Frame handling:

const MachineFrameInfo &MFI = MF.getFrameInfo(); // Line 97

So I run LLDB on the command from Step2 and provide the breakpoint at Line number 97 and provide full path to the AVR’s source code file (I found the command to specify breakpoint at a specific line number in a file through the internet):

$> cd /home/uhdesign/llvm_work/my_tests/
$> lldb /home/uhdesign/llvm_work/llvm_build/bin/clang -- --target=avr -S test_add.c 
(lldb) target create "/home/uhdesign/llvm_work/llvm_build/bin/clang"
Current executable set to '/home/uhdesign/llvm_work/llvm_build/bin/clang' (x86_64).
(lldb) settings set -- target.run-args  "--target=avr" "-S" "test_add.c"
(lldb) breakpoint set --file /home/uhdesign/llvm_work/llvm_download/llvm-project/llvm/lib/Target/AVR/AVRFrameLowering.cpp --line 97
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) 

The above step fails to place breakpoint :frowning:

Questions:
So can you please answer the following:
Q1) What am I doing wrong with Step3?

Q2) Is my whole approach (i.e. to learn a target’s source code by generating Assembly and using breakpoints at certain locations in source code) correct?

Q3) For some reason when I ran the commands that you provided for clang --version then instead of LLDB pointing me to the source code, it showed me some assembly dump (please see below). Any idea why that might be happening?? I would like LLDB to point me to source code similar to what you got (instead of showing assembly dump)

$> cd /home/uhdesign/llvm_work/my_tests/
$> lldb /home/uhdesign/llvm_work/llvm_build/bin/clang -- --version
(lldb) target create "/home/uhdesign/llvm_work/llvm_build/bin/clang"
Current executable set to '/home/uhdesign/llvm_work/llvm_build/bin/clang' (x86_64).
(lldb) settings set -- target.run-args  "--version"
(lldb) b Driver::PrintVersion
Breakpoint 1: where = clang`clang::driver::Driver::PrintVersion(clang::driver::Compilation const&, llvm::raw_ostream&) const [clone .localalias], address = 0x000000000384b548
(lldb) run
Process 28649 launched: '/home/uhdesign/llvm_work/llvm_build/bin/clang' (x86_64)
Process 28649 stopped
* thread #1: tid = 28649, 0x000000000384b548 clang`clang::driver::Driver::PrintVersion(clang::driver::Compilation const&, llvm::raw_ostream&) const [clone .localalias], name = 'clang', stop reason = breakpoint 1.1
    frame #0: 0x000000000384b548 clang`clang::driver::Driver::PrintVersion(clang::driver::Compilation const&, llvm::raw_ostream&) const [clone .localalias]
clang`clang::driver::Driver::PrintVersion:
->  0x384b548 <+0>: pushq  %rbp
    0x384b549 <+1>: movq   %rsp, %rbp
    0x384b54c <+4>: pushq  %rbx
    0x384b54d <+5>: subq   $0x128, %rsp              ; imm = 0x128 
(lldb) 

Thank you very much again for all your input and reading this with great patience :slight_smile:

Best Regards

This is a good tactic for sure.

One thing a lot of people ask for in llvm is some kind of “learning target” but the problem is, all targets, even made up ones, have weird quirks of some kind. So you’ll always have a somewhat incomplete picture if you only look at one target.

(not that you need a complete picture, some of these quirks are not worth knowing about unless you get paid to :slight_smile: )

These are not best/better/worse ways so if you learn what you need to learn then it’s up to you to decide if you will use that technique again. Printf is not worse than a debugger which is not worse than blinking an LED, they all have their places. You’ll get a sense for what works for you over time.

And remember that when you hear about how someone found X bug or explained Y process, you’re usually hearing the 1 thing that worked. Not the 10 things that didn’t, or they thought about but didn’t get to because the first thing they tried worked.

For embedded platforms like AVR, the standard libraries e.g. the C library can be built in very particular ways depending on what capabilities the target device has. This happens for Arm too, we have so many extensions you can have and your hardware might have a random selection of those.

So what clang is saying is, because you didn’t tell it what specific hardware this program is for, it doesn’t know what variant of the libraries to tell the linker to use.

That said, I don’t think this is a problem because you are only generating assembly, not linking a final binary.

If anything, I’d say this might in fact be a bug. Clang generally shouldn’t complain about lack of information for a linker that it isn’t going to call.

If you want to silence it, you can do another trick I use a lot. When asked to provide some option to a program, give it that option with a random value and if you’re lucky, it’ll reject it with a helpful list of possible values:

$ ./bin/clang --target=avr /tmp/test.c -S -o - -mmcu=dfsdfds
error: unknown target CPU 'dfsdfds'
note: valid target CPU values are: avr1, at90s1200, attiny11, attiny12, ...

For assembly I doubt it’s that important, so -mmcu=atmega4809 is probably fine.

Gonna try this locally and see what result I get…

Raised an issue for the warning: AVR rtlib linking quirks warning emitted even when not linking · Issue #96743 · llvm/llvm-project · GitHub

If it wasn’t this page - GDB to LLDB command map - 🐛 LLDB, that page is also a very useful reference.

I was able to breakpoint correctly locally:

$ ./bin/lldb ./bin/clang -- --target=avr -S /tmp/test_add.c -o - -mmcu=atmega4809
(lldb) target create "./bin/clang"
Current executable set to '/home/david.spickett/build-llvm-aarch64/bin/clang' (aarch64).
(lldb) settings set -- target.run-args  "--target=avr" "-S" "/tmp/test_add.c" "-o" "-" "-mmcu=atmega4809"
(lldb) breakpoint set --file llvm/lib/Target/AVR/AVRFrameLowering.cpp --line 97
Breakpoint 1: where = clang`llvm::AVRFrameLowering::emitPrologue(llvm::MachineFunction&, llvm::MachineBasicBlock&) const + 1492 at AVRFrameLowering.cpp:97:33, address = 0x0000000002d81aa4
(lldb) run
Process 11610 launched: '/home/david.spickett/build-llvm-aarch64/bin/clang' (aarch64)
        .text
.set __tmp_reg__, 0
.set __zero_reg__, 1
.set __SREG__, 63
.set __SP_H__, 62
.set __SP_L__, 61
        .file   "test_add.c"
Process 11610 stopped
* thread #1, name = 'clang', stop reason = breakpoint 1.1
    frame #0: 0x0000aaaaad82baa4 clang`llvm::AVRFrameLowering::emitPrologue(this=0x0000fffff7a94790, MF=0x0000aaaabf1c2300, MBB=0x0000aaaabf1c2c70) const at AVRFrameLowering.cpp:97:33
   94       return;
   95     }
   96
-> 97     const MachineFrameInfo &MFI = MF.getFrameInfo();
   98     unsigned FrameSize = MFI.getStackSize() - AFI->getCalleeSavedFrameSize();
   99
   100    // Skip the callee-saved push instructions.

So the first thing you should check is that the clang binary does in fact have debug symbols. ./bin/clang is actually a symlink, so go to where that points to to get the real binary:

$ file ./bin/clang
./bin/clang: symbolic link to clang-19
$ file ./bin/clang-19
./bin/clang-19: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, with debug_info, not stripped

You want to see with debug_info, not stripped. You should also see some delay in lldb when it starts up, as it processes the debug info. Though if you’re not used to debuggers this may or may not be noticeable.

The other thing that can happen is that there is debug info but the paths for the source files are recorded in such a way that lldb cannot find them. There’s a few causes for that but a general tip is to debug from the directory that you built the program in. That’s what I’ve done in the output I showed.

You can remap these paths using the target.source-map setting, but it’s best avoided if you can.

If none of that works, you can try building a minimal program with debug info and make sure that lldb can at least pick that up.

$ <the compiler you used to build llvm> /tmp/test.c -o /tmp/test.o -g
$ ./bin/lldb /tmp/test.o -o "b main" -o "run"
(lldb) target create "/tmp/test.o"
Current executable set to '/tmp/test.o' (aarch64).
(lldb) b mainly indexing DWARF: test.o...
Breakpoint 1: where = test.o`main at test.c:2:10, address = 0x000000000000071c
(lldb) run
Process 14113 launched: '/tmp/test.o' (aarch64)
Process 14113 stopped
* thread #1, name = 'test.o', stop reason = breakpoint 1.1
    frame #0: 0x0000aaaaaaaaa71c test.o`main at test.c:2:10
   1    int main() {
-> 2      return 0;
   3    }

There’s a slim chance that the gcc 13 you built is producing debug info in a way that the apt version of lldb cannot understand, and is silently dropping.

In which case you could try the compiler flag -gdwarf-4 (add this to CMAKE_CXX_FLAGS and CMAKE_C_FLAGS), or build a copy of a newer lldb, or try gdb instead.

Thank you very much. Yes now I am able to see source code as well.

Thanks :slight_smile:

Hi @DavidSpickett ,

Now that my GDB is running and I am able to see debug data I have one question about how to view contents of SmallVector in a more readable way? Is it even possible?

For example I have made the following test code in a file test_add.c:

int my_add(int a, int b)
{
   int c;
   c = a + b;
   return c;
}

Now I run GDB session for above test code:
gdb -args /home/uhdesign/llvm_work/llvm_build/bin/clang --target=avr -mmcu=atmega328 -S test_add.c

This gives me the GDB prompt where I set the following breakpoint and run :

b /home/uhdesign/llvm_work/llvm_download/llvm-project/llvm/lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp:99
r

This takes me to my breakpoint as expected:

Starting program: /home/uhdesign/llvm_work/llvm_build/bin/clang --target=avr -mmcu=atmega328 -S test_add.c
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, llvm::FunctionLoweringInfo::set (this=0x13528fa0, fn=..., mf=..., DAG=0x135294d0) at /home/uhdesign/llvm_work/llvm_download/llvm-project/llvm/lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp:101
101	  CanLowerReturn =
(gdb) list
96	  SmallVector<ISD::OutputArg, 4> Outs;
97	  CallingConv::ID CC = Fn->getCallingConv();
98	
99	  GetReturnInfo(CC, Fn->getReturnType(), Fn->getAttributes(), Outs, *TLI,
100	                mf.getDataLayout());
101	  CanLowerReturn =
102	      TLI->CanLowerReturn(CC, *MF, Fn->isVarArg(), Outs, Fn->getContext());
103	
104	  // If this personality uses funclets, we need to do a bit more work.
105	  DenseMap<const AllocaInst *, TinyPtrVector<int *>> CatchObjects;
(gdb)

Now here I am trying to see the contents of Outs which is of type SmallVector<ISD::OutputArg, 4> (defined in line 96).
So I do the following:

(gdb) p Outs
$1 = {<llvm::SmallVectorImpl<llvm::ISD::OutputArg>> = {<llvm::SmallVectorTemplateBase<llvm::ISD::OutputArg, true>> = {<llvm::SmallVectorTemplateCommon<llvm::ISD::OutputArg, void>> = {<llvm::SmallVectorBase<unsigned int>> = {BeginX = 0x7fffffff9290, Size = 1, Capacity = 4}, <No data fields>}, static TakesParamByValue = false}, <No data fields>}, <llvm::SmallVectorStorage<llvm::ISD::OutputArg, 4>> = {
    InlineElts = '\000' <repeats 12 times>, "\006\000\000\000\006\214\377\377", '\000' <repeats 12 times>, "\001\220\377\377", '\000' <repeats 12 times>, "\270\315v\f\000\000\000\000\021\000\000\000\001\000\000\000\250nQ\023\000\000\000\000@\223\377\377\377\177\000\000\020\223\377\377\377\177\000\000\376\t-\004\000\000\000\000PoQ\023\000\000\000\000@oQ\023\001\000\000\000\250nQ\023\000\000\000\000@\223\377\377\001\177\000\000\340nQ\023\000\000\000\000x\223\377\377\377\177\000\000P\223\377\377\377\177\000\000\000\224\377\377\377\177\000\000`\223\377\377\377\177\000\000\244#\315\002\000\000\000\000@oQ\023\000\000\000\000\000\224\377\377\377\177\000"}, <No data fields>}
(gdb)

In the above dump I do see that the size to be 1 (so there is some valid content in the vector). But it all seems gibberish.

So my question is:
Is there a way to see the content of Outs in a more meaningful/readable way?

Thank you again for all the extremely valuable inputs :slight_smile:

FYI, my Clang seems to have debug info not stripped:

$> file /home/uhdesign/llvm_work/llvm_build/bin/clang
/home/uhdesign/llvm_work/llvm_build/bin/clang: symbolic link to clang-19
$> file /home/uhdesign/llvm_work/llvm_build/bin/clang-19
/home/uhdesign/llvm_work/llvm_build/bin/clang-19: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, not stripped

Yeah, welcome to C++ debug :slight_smile:

You can add scripts to debuggers to tell them how to render certain types, instead of showing them “literally” which is what you’re seeing here.

We have a pretty printer script here llvm-project/llvm/utils/gdb-scripts/prettyprinters.py at main · llvm/llvm-project · GitHub.

I’ve not used it myself but judging by articles like https://undo.io/resources/gdb-watchpoint/here-quick-way-pretty-print-structures-gdb, you download that file and source <the file> to load it. And I presume you could add that to a gdbinit (gdbinit man (Debugging with GDB)) file to do it automatically. Then it’ll take over when it sees a type it recognises, and print it in a more useful way.

And you can find other scripts out there for the standard library types. libstdc++ probably has similar scripts.

Also remember that you can call functions within the debugger so if the print out is useless, but you know it acts like a vector you could do:

(gdb) p foo.size()
<some number N>
(gdb) p foo[0]
<some value>
(gdb) p foo[1]
<and so on...>

Look at the .h file for the type to get ideas for that, or the surrounding code.

And if you’ve got a container that uses a pointer to its storage you can dereference that:

(gdb) p *storage
<some actual values>

Pretty much anything the code would do.