Currently it always has to as on non-OSX platforms it is using
DWARFCompileUnit::Index(). But as I plan to implement DWARF-5 .debug_names
index (like _apple* index) maybe LLDB then no longer needs to populate
m_die_array and so just expanding all DW_TAG_partial_unit into a single
m_die_array for each DW_TAG_compile_unit is fine?
So I glossed over the documentation and I gathered that DWARF type info
might be stored in other DWARF files and references from the current file.
Yes. Arbitrary DIEs, not just type-defining DIEs. DW_TAG_imported_unit can
happen anywhere according to the DWARF standard but for the DWZ/Fedora case it
is a bit simpler one can limit the support for only DW_TAG_imported_unit with
parent of DW_TAG_compile_unit (or DW_TAG_partial_unit for nested imports).
DWZ ever uses DW_TAG_imported_unit only at the CU/PU top level.
SymbolFileDWARFDebugMap is an example of how we do things on MacOS. We have
one clang::ASTContext in the SymbolFileDWARFDebugMap, and multiple external
.o files (where each ins a SymbolFileDWARF instance) that contain unlinked
DWARF. Each SymbolFileDWARF instance will have:
void SymbolFileDWARF::SetDebugMapModule(const lldb::ModuleSP &module_sp);
called to indicate it is actually part of the SymbolFileDWARFDebugMap.
Then there are functions that check the debug map file and return the
UniqueDWARFASTTypeMap or the TypeSystem from the SymbolFileDWARFDebugMap if
we have one:
Therefore IIUC you make a single context for all types from a compiled
program? Primitive (non-class) types can be different across CUs:
CU1: typedef int foo_t;
CU2: typedef long foo_t;
We can and we do allow this. We will have two foo_t definitions in the AST context at the same level. Each variable has the correct type as it directly points to the right “foo_t”. When these variables are used in expressions, we will only copy 1 of the “foo_t” definitions over into the expression AST context so things normally work out. If you tried to cast something to a “foo_t” you would run into problems though since the type would be ambiguous.
I have a problem that one DW_TAG_partial_unit can be included by multiple
DW_TAG_compile_unit. Therefore DWARFCompileUnit for DW_TAG_partial_unit
itself cannot map itself by SymbolFileDWARFDebugMap to its parent
DWARFCompileUnit for DW_TAG_compile_unit (as it has multiple parents).
I expect you cannot link a single MacOS object file into two diferent
programs/libraries which are debugged at once by LLDB.
You can. We always open a new instance of the .o file, one for each SymbolFileDWARFDebugMap or unique executable that refers to the .o file.
One other idea is to keep all DWARF files separate and stand alone. Your
main DWARF file with one or more DW_TAG_imported_unit and all
DW_TAG_imported_unit referenced files, each as its own SymbolFileDWARF. Any
reference to a DW_FORM_ref_alt would turn into a forward declaration in the
current SymbolFileDWARF, so the ASTContext in each SymbolFileDWARF wouldn’t
know anything about the types,
Is it applicable even if DW_TAG_imported_unit points to DW_TAG_partial_unit’s
containing DW_TAG_variable, DW_TAG_subprogram and arbitrary other DIEs, not
just types?
This approach will not work if that is the case. Scratch the above idea…
The other approach I might suggest is to write a DWARF linker, maybe using
LLVM’s DWARF classes (see llvm-dsymutil sources) that takes the top level
DWARF and all DW_TAG_imported_unit files and combines them all back into one
large DWARF file.
That would defeat the advantage of DWZ-optimized debug info files. DWZ
reduces their size by approx. 30%. If the relinking was made on-demand then
it defeats the LLDB performance advantage over GDB - GDB can already read DWZ
natively (GDB does duplicates the internal representation when reading DWZ
CUs/PUs). In such case I could already code such reconstruction of non-DWZ
debug info when reading m_die_array - but that also seems needlessly slow to
me. DWZ should bring even performance advantage of parsing the DWZ common
file (=imported file) only once.
One other idea is to let each DWARF file be separate, and when you need
a type from a DW_TAG_imported_unit you log that file as stand alone and copy
the type from its clang::ASTContext into the main SymbolFileDWARF’s AST
context. We copy types all the time in expressions as each on has its own
AST context.
Does this work even for non-type DIEs?
No it does not. Only types are fine, but anything else will fall down.
So there are many solutions. I would vote for linking the DWARF into
a single file much like we do with llvm-dsymutil on Mac, but that really
depends if the type uniquing is desired within a single DWARF file and not
across many shared libraries that all reference common DW_TAG_imported_unit
files.
I agree the DWZ optimization does primarily what -fdebug-types-section does.
I do not see why to really do relinking files on disk, debugger should not
need that to read the debug info.
I do not fully understand why you do the llvm-dsymutil relinking when you
already have SymbolFileDWARFDebugMap in LLDB. But I do not know OSX/Mac.
So this is a new and unique case that will require substantial support. Not sure what the best approach is. Symbol files get to pick the IDs of their lldb_private::CompileUnit (which is the DW_TAG_compile_unit DIE offset for normal DWARF), lldb_private::Function (which is the DW_TAG_subprogram DIE offset), lldb_private::Block (which is the DW_TAG_lexical_block and DW_TAG_inlined_subroutine DIE offset), and lldb_private::Type (type DIE offset).
You might need to see what they did for the DWO support where they overload stuff in the DIERef class. See DIERef.cpp. Basically you must ensure that your ID for the things you create in the DWARF parser and hand out to LLDB can easily be used to find the original DIE. DIERef objects must be able to make unique lldb::user_id_t values for each item and given a lldb::user_id_t you must be able to get back to the compile unit, function, block, type etc very efficiently. This space is crowded now that we have normal DWARF, debug map DWARF (SymbolFileDWARFDebugMap) on MacOSX, DWO (https://gcc.gnu.org/wiki/DebugFission) support on other systems, and DWZ on your system. You will need to build in your DWZ support without affecting all of the others that are in place.
The test suite is pretty good for this. I have a patch that attempt to add -fdebug-types-support here:
https://reviews.llvm.org/D32167
The gist of this patch is since we know that .debug_types has no references to any other compile units or type units, we can pretend that the ID for anything in .debug_types is just sizeof(.debug_info) + .debug_types DIE offset. This patch also shows how to have the test suite add -fdebug-types-section to the LLDB test suite to ensure it is tested. Each code test in the LLDB test suite can be duplicated and run with each different kind of debug info flavor. I haven’t had time to work out all of the issues in this patch, but it is close.
DWZ is trickier in that it really does want things to be shared between files, including functions, types, and so much more. If you end up trying to have each DWARF file stay separate, you are going to be copying a bunch of stuff from one file’s AST context to the other. We have type copying support currently, but I am not sure of the support for variable and function AST copying.
The easiest solution would be to load the main DWARF file, all of its lldb::user_id_t value would just be the normal .debug_info offset. For any files that are mentioned via DW_TAG_imported_unit, their IDs would be the previous .debug_info size + the offset in the file:
dwarf[0] a.out (main DWARF file) lldb::user_id_t is just the actual DIE offset
dwarf[1] /tmp/shared1 (DW_TAG_imported_unit from a.out) lldb::user_id_t is just the DIE offset + sizeof(dwarf[0] .debug_info)
dwarf[2] /tmp/shared2 (DW_TAG_imported_unit from a.out) lldb::user_id_t is just the DIE offset + sizeof(dwarf[0] .debug_info) + sizeof(dwarf[1] .debug_info)
The drawback is this won’t allow sharing /tmp/shared1 or /tmp/shared2 between two different top level DWARF files, but it does allow one clang::ASTContext to be used between all of them. SymbolFileDWARFDebugMap makes it lldb::user_id_t contain the CU index in the top 32 bits, and the DIE offset within that .o file’s DWARF in the bottom 32 bits. You could do something similar in your case where the top 32 bits is the index of the DWARF file in the “dwarf” array that would be maintained in a new SymbolFileDWARFDWZ subclass.