ARM AARCH64 and AARCH32 in the same debugger

Here’s a scenario I’m being asked about:

A CPU is executing in AARCH32. LLDB is talking to a JTAG probe via gdb remote. The user stops the program, and it’s in an Android user space application. Can we single step through it, and then an AARCH64 Hypervisor routine, in the same LLDB instance? What would we need to set up to do this?

Yes you can: you can have one target for the 32 bit ARM and one for the 64 bit one. You currently can't have one target (process) be both 32 and 64 bit, but you can do this with two targets. I am guessing you would have one target that is debugging the user space process and one for the 64 bit hypervisor routine which I am assuming is the kernel?

This is exactly why we developed LLDB with a modular plug-in based architecture.


I'm guessing the JTAG probe vends a single gdb-remote connection that is either seeing the user-process aarch32 or the hypervisor aarch64 at any given time. So you're stepping along in aarch32 mode and then it syscalls out to aarch64, all within a single Target.

I think this is akin to the folks doing the low-level startup routines on x86 machines - it changes its architecture from 32-bit to 64-bit as it brings up the system. From lldb's point of view this is a single Target that changes architecture during the debug session.

FWIW we were talking about this a little bit. Assuming I'm correct that you have one communication channel to the JTAG probe (one gdb-remote connection), when you step from aarch32 user process code to aarch64 hypervisor code it's not just an architecture change -- we're changing to an entirely different program (kernel versus user code).

This genuinely does need to be represented in lldb as two separate Targets that coexist simultaneously. Only one of them will be executing at any given time - so there would need to be some way for lldb to hand off from userland Target to kernel Target. That's definitely something new.

If there are separate probes/gdb-remote communication channels for these -- maybe the kernel is running on one core and the user process on another -- then it's easy.

The EFI example I mentioned earlier where a processor changes its mode (16/32, then 64 bit) as it starts up is not a good comparison. In that case a single program is being debugged through the entire transition - it's just the architecture that has changed mid-debug-session.


One of the issues here is the ability to capture the context of the current debugging situation, and it depends on a whole lot of variables.

With JTAG gdbservers it may not care that much about the difference in programs at the developer/process/OS level. It may simply provide static single thread per core. There is not enough information here, but, in this instance, it can be represented with a single target to be seen by LLDB. LLDB will have to do the heavy lifting at this point to switch archs and locate 'real' thread information which we have done in the past with success. In this mode, the user at least gets a single coherent view of what is being debugged. Single stepping from one 'arch' to another can be possible seamlessly, given a good implementation.

The other situation, which is mentioned before, uses multiple targets. This works correctly if you have multiple gdbservers for the separate bits of what is being debugged, although, from a developer standpoint, things get complicated:

1) There is no 'high level view' of the current debugging situation at any present time.
2) There is no cooperation between the targets, _even_ if they share resources like memory.
3) Single stepping from one target to the other is not seamless.
4) Events can be missed from one target whilst debugging the other.
5) goes on..

This is why last year I suggest we start preparing for some sort of higher-level platform which can manage multiple targets in a way which makes sense for a given architecture/soc/system/whatever. It can understand memory can be shared between targets, and therefore watchpoints can trigger events on different platforms. It can understand in a single stepping situation there may be a need to switch targets given a set of circumstances to allow for a seamless experience to developers.

At this point, whilst multiple target debugging works, it's value can be so much more. Especially given the targets LLDB is tending to support; parts of larger SoCs.