DW_TAG_member extends beyond the bounds error on Linux

Hi,

While dogfooding our lldb based IDE on Linux, I am seeing a lot of variable evaluation errors related to DW_TAG_member which prevents us from release the IDE. Can anyone help to confirm if they are known issues? If not, any information you need to troubleshoot this issue?

Here is one example:

(lldb) fr v
error: biggrep_master_server_async 0x10b9a91a: DW_TAG_member ‘_M_pod_data’ refers to type 0x10bb1e99 which extends beyond the bounds of 0x10b9a901
error: biggrep_master_server_async 0x10b98edc: DW_TAG_member ‘small_’ refers to type 0x10bb1d9f which extends beyond the bounds of 0x10b98ed3
error: biggrep_master_server_async 0x10baf034: DW_TAG_member ‘__size’ refers to type 0x10baf04d which extends beyond the bounds of 0x10baefae
(facebook::biggrep::BigGrepMasterAsync *) this = 0x00007fd14d374fd0
(const string &const) corpus = error: summary string parsing error: {
store_ = {
= {
small_ = {}
ml_ = (data_ = “��UH\x89�H�}�H\x8bE�]ÐUH\x89�H��H\x89}�H\x8bE�H\x89��~\xb4��\x90��UH\x89�SH\x83�H\x89}�H�u�H�E�H���\x9e���H\x8b\x18H\x8bE�H���O\xb4��H\x89ƿ\b”, size_ = 0, capacity_ = 1441151880758558720)
}
}
}
(const string &const) needle = error: summary string parsing error: {
store_ = {
= {
small_ = {}
ml_ = (data_ = “”, size_ = 0, capacity_ = 1080863910568919040)
}
}
}
(facebook::biggrep::Options &) options = 0x00007fd133cfb7b0: {
engine = error: summary string parsing error
full_lines = true
user = error: summary string parsing error
max_bytes = 5000000
leading_context = 0
trailing_context = 0
case_sensitive = true
client_hostname = error: summary string parsing error
client_ip = error: summary string parsing error
skip_logging = false
client_port = 0
shards_override = 0
sample = false
count = false
filename_pattern = error: summary string parsing error
limit = 0
isset = {
engine = true
full_lines = true
user = true
max_bytes = true
leading_context = true
trailing_context = true
case_sensitive = true
client_hostname = true
client_ip = true
skip_logging = true
client_port = true
shards_override = true
sample = true
count = true
filename_pattern = true
limit = true
}
}
(size_t) recv_timeout = 140536468041728
(std::vector<std::basic_fbstring<char, std::char_traits, std::allocator, std::fbstring_core >, std::allocator<std::basic_fbstring<char, std::char_traits, std::allocator, std::fbstring_core > > >) corpuses = size=0 {}
(std::vector<facebook::biggrep::BigGrepMasterAsync::Revision, std::allocatorfacebook::biggrep::BigGrepMasterAsync::Revision >) revisions = size=0 {}
(std::vector<facebook::biggrep::BigGrepMasterAsync::RevisionShard, std::allocatorfacebook::biggrep::BigGrepMasterAsync::RevisionShard >) shards = size=0 {}
(std::string) returnRev = error: summary string parsing error
(<lambda(folly::StringPiece)>) quote = {}
(std::basic_fbstring<char, std::char_traits, std::allocator, std::fbstring_core >) desc = {
store
= {
= {
small
= {}
ml_ = (data_ = “”, size_ = 73415312, capacity_ = 140536494141696)
}
}
}
(folly::EventBase *) eb = 0x00007fd133cfb888
(apache::thrift::concurrency::ThreadManager *) tm = 0x00007fd133cfb570

I suspect each one may be different root cause. I was able to capture one callstack of a small repro:

Breakpoint 1, DWARFASTParserClang::ParseChildMembers (this=0x8c4520, sc=…, parent_die=…, class_clang_type=…, class_language=lldb::eLanguageTypeUnknown,

base_classes=…, member_accessibilities=…, member_function_dies=…, delayed_properties=…,
default_accessibility=@0x7ffdf3888cac: lldb::eAccessPublic, is_a_class=@0x7ffdf3888cab: false, layout_info=…)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp:2937
2937 parent_die.GetID());
(gdb) bt
#0 0x00007f103d02533d in DWARFASTParserClang::ParseChildMembers(lldb_private::SymbolContext const&, DWARFDIE const&, lldb_private::CompilerType&, lldb::LanguageType, std::vector<clang::CXXBaseSpecifier*, std::allocatorclang::CXXBaseSpecifier* >&, std::vector<int, std::allocator >&, DWARFDIECollection&, std::vector<DWARFASTParserClang::DelayedAddObjCClassProperty, std::allocatorDWARFASTParserClang::DelayedAddObjCClassProperty >&, lldb::AccessType&, bool&, DWARFASTParserClang::LayoutInfo&) (this=0x8c4520, sc=…, parent_die=…, class_clang_type=…, class_language=lldb::eLanguageTypeUnknown, base_classes=…, member_accessibilities=…, member_function_dies=…, delayed_properties=…, default_accessibility=@0x7ffdf3888cac: lldb::eAccessPublic, is_a_class=@0x7ffdf3888cab: false, layout_info=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp:2937
#1 0x00007f103d025b84 in DWARFASTParserClang::CompleteTypeFromDWARF(DWARFDIE const&, lldb_private::Type*, lldb_private::CompilerType&) (this=0x8c4520, die=…, type=0xc40a50, clang_type=…)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Plugins/SymbolFile/DWARF/DWARFASTParserClang.cpp:2036
#2 0x00007f103d04c5e8 in SymbolFileDWARF::CompleteType(lldb_private::CompilerType&) (this=, compiler_type=…)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp:1635
#3 0x00007f103ceff9da in lldb_private::ClangASTContext::CompleteTagDecl(void*, clang::TagDecl*) (baton=, decl=0xc41bc0)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Symbol/ClangASTContext.cpp:9619
#4 0x00007f103cf03400 in GetCompleteQualType(clang::ASTContext*, clang::QualType, bool) (ast=0x7f102c020c30, qual_type=…, allow_completion=allow_completion@entry=true) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Symbol/ClangASTContext.cpp:2632
#5 0x00007f103cf0c99e in lldb_private::ClangASTContext::GetNumChildren(void*, bool) (this=0x8c43c0, type=, omit_empty_base_classes=) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Symbol/ClangASTContext.cpp:5067
#6 0x00007f103ce876ad in lldb_private::ValueObjectVariable::CalculateNumChildren(unsigned int) (this=, max=4294967295)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Core/ValueObjectVariable.cpp:102
#7 0x00007f103ce7759f in lldb_private::ValueObject::GetNumChildren(unsigned int) (this=0xc40600, max=4294967295)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Core/ValueObject.cpp:778
#8 0x00007f103cd999bd in lldb_private::FormatManager::ShouldPrintAsOneLiner(lldb_private::ValueObject&) (this=this@entry=0x7f103fe2f300 <GetFormatManager()::g_format_manager>, valobj=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/DataFormatters/FormatManager.cpp:576
#9 0x00007f103cd9226a in lldb_private::DataVisualization::ShouldPrintAsOneLiner(lldb_private::ValueObject&) (valobj=…)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/DataFormatters/DataVisualization.cpp:42
#10 0x00007f103cdc2167 in lldb_private::ValueObjectPrinter::PrintChildrenIfNeeded(bool, bool) (this=this@entry=0x7ffdf38892a0, value_printed=, summary_printed=) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/DataFormatters/ValueObjectPrinter.cpp:869
#11 0x00007f103cdc1572 in lldb_private::ValueObjectPrinter::PrintValueObject() (this=this@entry=0x7ffdf38892a0)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/DataFormatters/ValueObjectPrinter.cpp:111
#12 0x00007f103ce7102f in lldb_private::ValueObject::Dump(lldb_private::Stream&, lldb_private::DumpValueObjectOptions const&) (this=this@entry=0xc40600, s=…, options=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Core/ValueObject.cpp:3585
#13 0x00007f103d27e5e4 in CommandObjectFrameVariable::DoExecute(lldb_private::Args&, lldb_private::CommandReturnObject&) (this=0x7d0590, command=…, result=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Commands/CommandObjectFrame.cpp:591
#14 0x00007f103cece45d in lldb_private::CommandObjectParsed::Execute(char const*, lldb_private::CommandReturnObject&) (this=0x7d0590, args_string=, result=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Interpreter/CommandObject.cpp:1095
#15 0x00007f103cec9559 in lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&, lldb_private::ExecutionContext*, bool, bool) (this=this@entry=0x7a5210, command_line=, lazy_add_to_history=lazy_add_to_history@entry=lldb_private::eLazyBoolCalculate, result=…, override_context=override_context@entry=0x0, repeat_on_empty_command=repeat_on_empty_command@entry=true, no_context_switching=false)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Interpreter/CommandInterpreter.cpp:1820
#16 0x00007f103ceca50d in lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::string&) (this=0x7a5210, io_handler=…, line=…) at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Interpreter/CommandInterpreter.cpp:2976
#17 0x00007f103ce1c733 in lldb_private::IOHandlerEditline::Run() (this=0x87d9b0)
at /home/engshare/third-party2/lldb/3.8.0.rc3/src/llvm/tools/lldb/source/Core/IOHandler.cpp:736

Really appreciate if there is any fix/workaround I can get over this issue and unblock us!

Jeffrey

Sorry, sent to the wrong alias.

If you’re going to use clang built binaries with lldb, you’ll want to pass -fstandalone-debug - this is the default on platforms where lldb is the primary debugger (Darwin and freebsd)

Not sure if that is the problem you are seeing, but will be a problem sooner or later

Thanks David. I meant to send to lldb maillist, but glad to hear response here.

Our binary is built from gcc:
String dump of section ‘.comment’:
[ 1] GCC: (GNU) 4.9.x-google 20150123 (prerelease)

Is there any similar flags we should use? By doing “strings -a [binary] | grep -i gcc”, I found the following flags being used:
GNU C++ 4.9.x-google 20150123 (prerelease) -momit-leaf-frame-pointer -m64 -mtune=generic -march=x86-64 -g -O3 -O3 -std=gnu++11 -ffunction-sections -fdata-sections -fstack-protector -fno-omit-frame-pointer -fdebug-prefix-map=/home/engshare/third-party2/icu/53.1/src/icu=/home/engshare/third-party2/icu/53.1/src/icu -fdebug-prefix-map=/home/engshare/third-party2/icu/53.1/src/build-gcc-4.9-glibc-2.20-fb/no-pic=/home/engshare/third-party2/icu/53.1/src/icu -fno-strict-aliasing --param ssp-buffer-size=4

Also, per reading https://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/Debugging-Options.html, seems that we should use “-gdwarf-2” to generate only standard DWARF format? I think I might need to chat with our build team but want to know which flag I need ask them first.

Btw: I tried gdb against the same binary which seems to get better result:

(gdb) p corpus
$3 = (const std::string &) @0x7fd133cfb888: {
static npos = 18446744073709551615, store_ = {
static kIsLittleEndian = ,
static kIsBigEndian = , {
small_ = “www”, ‘\000’ <repeats 20 times>, “\024”, ml_ = {
data_ = 0x777777 <std::Any_data::M_access<void folly::fibers::Baton::waitFiber<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}, void>::type::value_type folly::fibers::await<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1}>(folly::fibers::FiberManager&, folly::fibers::FirstArgOf<folly::fibers::FirstArgOf<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}, void>::type::value_type folly::fibers::await<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::{lambda(folly::fibers::Promise<facebook::servicerouter::RequestDispatcherBasefacebook::servicerouter::ThriftDispatcher::prepareForSelection(facebook::servicerouter::DispatchContext&)::SelectionResult>)#1}>(folly::fibers::FirstArgOf&&)::{lambda()#1}, void>::type::value_type)::{lambda(folly::fibers::Fiber&)#1}*>() const+25> “\311\303UH\211\345H\211}\370H\213E\370]ÐUH\211\345H\203\354\020H\211}\370H\213E\370H\211\307\350~\264\312\377\220\311\303UH\211\345SH\203\354\030H\211}\350H\211u\340H\213E\340H\211\307\350\236\377\377\377H\213\030H\213E\350H\211\307\350O\264\312\377H\211ƿ\b”, size = 0,
capacity
= 1441151880758558720}}}}

Jeffrey

Thanks David. I meant to send to lldb maillist, but glad to hear response
here.

Our binary is built from gcc:
String dump of section '.comment':
  [ 1] GCC: (GNU) 4.9.x-google 20150123 (prerelease)

Is there any similar flags we should use?

If it's the sort of issue I'm guessing it might be (though I have very
little evidence to go on/I don't remember the possible/specific failure
modes, etc) you might try -femit-class-debug-always with GCC

That said, to provide a more accurate diagnosis/help, a reduced test case
would be really helpful (the smallest C++ input (counting headers, etc, as
well - tools like creduce/delta/multidelta can help reduce test cases,
though they're better on compiler crashes than trying to preserve a running
program, etc) that produces the problem)

Is it possible for me to see the "biggrep_master_server_async" binary? I would like to look at the DWARF and try to figure out what is wrong. It seems that this code is what is causing the problem:

CompilerType member_clang_type = member_type->GetLayoutCompilerType ();
if (!member_clang_type.IsCompleteType())
    member_clang_type.GetCompleteType();

{
    // Older versions of clang emit array[0] and array[1] in the same way (<rdar://problem/12566646>).
    // If the current field is at the end of the structure, then there is definitely no room for extra
    // elements and we override the type to array[0].

    CompilerType member_array_element_type;
    uint64_t member_array_size;
    bool member_array_is_incomplete;

    if (member_clang_type.IsArrayType(&member_array_element_type,
                                      &member_array_size,
                                      &member_array_is_incomplete) &&
        !member_array_is_incomplete)
    {
        uint64_t parent_byte_size = parent_die.GetAttributeValueAsUnsigned(DW_AT_byte_size, UINT64_MAX);

        if (member_byte_offset >= parent_byte_size)
        {
            if (member_array_size != 1 && (member_array_size != 0 || member_byte_offset > parent_byte_size))
            {
                module_sp->ReportError ("0x%8.8" PRIx64 ": DW_TAG_member '%s' refers to type 0x%8.8" PRIx64 " which extends beyond the bounds of 0x%8.8" PRIx64,
                                        die.GetID(),
                                        name,
                                        encoding_form.Reference(),
                                        parent_die.GetID());
            }

            member_clang_type = m_ast.CreateArrayType(member_array_element_type, 0, false);
        }
    }
}

So we are parsing a member variable, and the member variable's type is an array type, and the offset of the member is beyond the parent's type's size and yet the array claims to have a valid size.

This code is trying to correctly create an CompilerType for a struct such as this:

struct MyStruct
{
    size_t num_entries;
    Entry entries[0];
};

Older versions of clang would emit the same DWARF for:

struct MyStruct
{
    size_t num_entries;
    Entry entries[0];
};

or

struct MyStruct
{
    size_t num_entries;
    Entry entries[1];
};

This is bad because in the DWARFASTParserClang we are trying to faithfully re-create the type as the user initially created it.

This error is trying to say that you have a structure where the byte size of "MyStruct" is something like 8 bytes, but "entries" is an array that has a valid size (not zero or one) and that the offset of "entries" is greater than or equal to 8. If "entries" has a size like:

struct MyStruct
{
    size_t num_entries;
    Entry entries[1];
};

You would expect the member offset for "entries" to be less than or equal to the sizeof(MyStruct). If "Entry" is 8 bytes in size, you might expect the byte size of "MyStruct" to be 16, and the offset of "num_entries" to be 0 and "entries" to be 8. Both offsets are less than or equal to the byte size of 16.

If I can see the DWARF file I can tell you more. If I can't, you will need to step through this code and see why this is triggering. The error message is telling you all you need to know to dump the problem. So for this error message:

error: biggrep_master_server_async 0x10b9a91a: DW_TAG_member '_M_pod_data' refers to type 0x10bb1e99 which extends beyond the bounds of 0x10b9a901

So if you dump the DWARF and look for the DIE at 0x10b9a91a, you will be able to see the DWARF that is causing the problem. On MacOSX, I would do this:

% dwarfdump --debug-info=0x10b9a91a --show-children --show-parents biggrep_master_server_async

llvm-dwarfdump will dump your entire file and you will need to look through a ton of DWARF in the output. You will need to search for "0x10b9a91a:" (the offset followed by a colon).

So LLDB believes it is pointing out an error in the DWARF that needs to be fixed in the compiler. It could be an incorrect error, but I would need to see the DWARF in question to know more. Debugging can happen just fine if you ignore this error, it is actually more of a warning, but we emit an error so users know to file a bug and hopefully get the compiler fixed.

Greg

This seems to have been fixed with the http://reviews.llvm.org/D18008.

Thanks Greg. Yes, as I tried in another thread, the error seems to go away after patching with this fix.