Yeah… that’s the issue I was encountering when I looked at it before my PTO. Will try to circle back to it this week.
1 Like
https://reviews.llvm.org/D137657
Just the parsing diff. Working on second one that changes internal representation of {cu,tu}-index offset to 64bit.
@dblaikie can you take a look at ⚙ D139379 [llvm][dwwarf] Change CU/TU index to 64-bit when you get a chance? 
Would love to land it and other diff before EOY.
Also previously you mentioned you were having issues with .debug_str.dwo overflowing 4GB in DWP. Did you make any progress on the mitigation effort, or have suggestions on how to address it?
Yeah, I’ll take a look at the review - sorry it dropped through the cracks for a bit .
Yeah, we’re addressing that with -gsimple-template-names (this emits DW_AT_name for most templates as the base name, without textual template arguments - relying on the DW_TAG_template_*_parameter DIEs to carry that information alone) which I’ve implemented in Clang and @aeubanks has implemented support/fixes for in LLDB recently. We’re about ready to turn that on at Google.
@pogo59 has proposed/accepted upstream that .debug_str_offsets.dwo could use DWARF64 where the rest of the dwo/dwp uses DWARF32, so that’d be an option as well, though it hasn’t been implemented - it could even be done selectively per-.debug_str_offsets.dwo contribution, if it contains overflowed strings, which would be relatively low cost in terms of size increase of DWP files.
Same thing.
1 Like
@dblaikie Gentle ping on ⚙ D139379 [llvm][dwwarf] Change CU/TU index to 64-bit.
I realize it’s holiday season, and you might be busy with other stuff. Is there anyone else that I can tag to review, if you are busy?
Do we have the same 4G limit with the dwp generated by GNU dwp tool ?
I hit the case that gdb complain the following error message with split-dwarf that build with clang + llvm-dwp/dwp. I am not sure if it is the same case.
Reading symbols from xxx …
Dwarf Error: wrong version in compilation unit header (is 718, should be 2, 3, 4 or 5) [in module xxx.dwp]
The size of xxx.dwp is over 7G.
GDB read the dwarf header from xxx.dwp and get incorrect version.
Sounds like it. GDB is probably trying to parse overflown offset, and interpreting some random piece of data as version. Side note. I and @clayborg are trying to add 64bit support to LLDB: ⚙ D138618 [LLDB] Enable 64 bit debug/type offset
That’s great! How about the gdb … We are still using gdb.
@dblaikie I know you mentioned your company hit issue with .debug_offset.dwo over 4GB. Can I ask how come you are not hitting it with .debug_info.dwo?
Yeah, I think initially it was just the type of code that lead to these issues - the tensorflow and Eigen expression templates that I think we’re especially string heavy - but as it happens I’ve been looking into a big this week that seems to be related to exceeding the debug_info.dwo limits, so that’s something we have in common now
1 Like
Hi, I am here with @zhangqingshan.zll . We presently encountered a case where a section offset overflowed when building a huge project. The overflow occurs at llvm-project/llvm-dwp.cpp at release/11.x · llvm/llvm-project · GitHub .
We propose a solution to fail fast and dump info when overflow occurs. Would this be helpful?
if (
__builtin_add_overflow(
(CurEntry.Contributions[Index].Length = Contents.size()),
ContributionOffsets[Index], &ContributionOffsets[Index])
) {
return make_error<DWPError>(std::string("section ") + std::to_string(Index) + std::string(" size overflow"));
}
There is a check in trunk for .debug_info.dwo
If you want you can add one for .debug_str.dwo
In theory .debug_types.dwo can overflow also, but since in DWARF5 type sections are moved to .debug_info.dwo and DWARF5 is default now it’s less of a concern I think.
1 Like
I posted a patch ⚙ D144565 dwp check overflow. This is supposed to backport and merge into branch release/11.x. This is my first time posting patches for llvm. Please help me get things right if I did not. 
@dblaikie I think I remember you mentioning somewhere that Google is using GDB. Is that true or am I miss remembering?
If that is the case do you know what version, and also if gdb-index + fdebug-type-sections is used with DWARF5 binaries?
There is a known bug in GDB (I don’t think it was fixed), where it crashes on binaries with gdb-index + fdebug-types-sections + cross CU references.
We’re vaguely migrating to lldb, but we do still have gdb kicking around.
We probably don’t hit an issue with “gdb-index + fdebug-types-sections + cross CU references” because we don’t use full LTO (that produces cross-CU references) only ThinLTO (which kind of produces cross-CU references… sort of - by creating little fragment CUs to reference from/to) but we use Split DWARF, which doesn’t support cross-CU References anyway, so LLVM skirts that issue by ThinLTO+Split DWARF just importing the cross-CU entities into the main CU, not using any cross-CU references.