Help with Xcode crash on lldb_private::(anonymous namespace)::parse_builtin_type

Hi Jim,

Thank you for the response!! I hadn’t subscribed to the lldb-dev mailing list until now, so I did not receive your response until googling for it :stuck_out_tongue:

Anyways, between when I sent my email and seeing your response, I tried very similar things to what you mentioned. I built lldb from latest source and replaced LLDB.framework in one of my Xcode versions with the version I built, adding some of my own logging, and also turned on demangle logging (I now know that I could have just used Xcode 8).

I am seeing some interesting results. To give a bit more background on the problem, let me explain it in a simplified version. Imagine we have 200 unit test executables all with their own main(), cross-compiled for iOS arm64. I put these into generated Xcode projects, and we use the Xcode projects to “build and run”. This allows Xcode to launch an executable as an app on an iOS device. Now imagine that each executable is a unit test for an iOS static library, so in this scenario we have 200 static libraries. Let’s say they are all in a DAG, so library 200 depends on library 199 and library 199 depends on library 198, etc. About halfway through the DAG, e.g. library 100, the demangler starts to crash. In this example, every unit test executable downstream of library 100 and including library 100 are causing the demangler to crash. It seems out best bet here is to find the offending symbol(s) and attempt to replace the code in our codebase with something the demangler can handle.

We have run the demangle log with several different executables and the demangler seems to crash on similar but slightly different symbols each time (of course, the symbol the demangler appears to be crashing on isn’t necessarily the offending symbol).

Using lldb from Xcode 7.3.1, I set Xcode 7.3.0 as my target executable and reproduced the crash, which allowed me to see the EXC_BAD_ACCESS in the debugger (and since I have Xcode 7.3.0 using my locally built version of lldb, I get to see lldb source information :p) :

Process 30313 stopped

  • thread #6: tid = 0xe0d1a, 0x00000001169d19b1 LLDB`char const* lldb_private::(anonymous namespace)::parse_expr_primary<lldb_private::(first=“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”, last="", db=0x0000700000216a58)::Db>(char const*, char const*, lldb_private::(anonymous namespace)::Db&) + 97 at CxaDemangle.cpp:2690, queue = ‘DBGLLDBLauncher Serial Queue’, stop reason = EXC_BAD_ACCESS (code=2, address=0x70000019cd98)

frame #0: 0x00000001169d19b1 LLDB`char const* lldb_private::(anonymous namespace)::parse_expr_primary<lldb_private::(first=“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”, last="", db=0x0000700000216a58)::Db>(char const*, char const*, lldb_private::(anonymous namespace)::Db&) + 97 at CxaDemangle.cpp:2690

2687 {

2688 if (last - first >= 4 && *first == ‘L’)

2689 {

→ 2690 switch (first[1])

2691 {

2692 case ‘w’:

2693 {

(lldb) quit

This the symbol from the above crash:

Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_

If I run c++filt, it just returns the mangled name back to me:

[sburke-maci:/Users/sburke]2 % c++filt Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_

Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_

Something similar happens if I run __cxa_demangle directly against the symbols we are investigating, including the one above. The demangler does not crash but the symbol is considered an “invalid mangled name” and cannot be demangled.
e.g.:

#include

#include “cxxabi.h"

int main(int argc, const char * argv[]) {

const char *bad_symbol1 =

“__ZZN5boost9function4IbRNSt3__111__wrap_iterIPKcEERKS5_RNS_6spirit7contextINS_6fusion4consIRN2fl3log8internal16enable_spec_elem13compound_maskENSB_4nil_EEENSB_7vector0IvEEEERKNS9_2qi10char_classINS9_3tag9char_codeINSR_5spaceENS9_13char_encoding5asciiEEEEEE9assign_toINSP_6detail13parser_binderINSP_4listINSP_11alternativeINSC_INSP_6actionINSP_9referenceINSP_7symbolsIcjNSP_3tstIcjEENSP_16tst_pass_throughEEEEENS_7phoenix5actorINS_5proto7exprns_10basic_exprINS1G_6tagns_3tag17bitwise_or_assignENS1G_7argsns_5list2INS1F_INS1I_INS1E_3tag4at_cENS1N_INS1I_INS1K_8terminalENS1M_4termIN4mpl_4int_ILi0EEEEELl0EEENS1F_INS9_9attributeILi0EEEEEEELl2EEEEENS1F_INS9_8argumentILi0EEEEEEELl2EEEEEEENSC_INS16_INSP_14literal_stringIRA5_S3_Lb1EEENS1F_INS1I_INS1K_6assignENS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IiEELl0EEEEEEELl2EEEEEEENSC_INS16_INS2A_IRA4_S3_Lb1EEENS1F_INS1I_IS2E_NS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IjEELl0EEEEEEELl2EEEEEEENSC_INS16_INS17_IKNSP_4ruleIS5_FjvENS1H_4exprIS1Q_NS1R_ISW_EELl0EEENS9_11unused_typeES31_EEEENS1F_INS1I_IS2E_NS1N_INS1I_IS1P_NS1N_INS1I_IS1Q_NS1R_INS1T_ILi1EEEEELl0EEES1Z_EELl2EEES25_EELl2EEEEEEESJ_EEEEEEEEEENSP_12literal_charINSU_8standardELb1ELb0EEEEENS1S_5bool_ILb0EEEEEEEvT_E13stored_vtable”;

const char *bad_symbol2 =

“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”;

size_t length = 0;

int status = 0;

//0: The demangling operation succeeded.

//-1: A memory allocation failiure occurred.

//-2: mangled_name is not a valid name under the C++ ABI mangling rules.

//-3: One of the arguments is invalid.

char *demangleChars = __cxxabiv1::__cxa_demangle(bad_symbol2, nullptr, &length, &status);

std::cout << "status = " << status << std::endl;

std::cout << "length = " << length << std::endl;

if(demangleChars) {

std::cout << demangleChars << std::endl;

free(demangleChars);

}

return 0;

}

This gives:

status = -2

length = 0

The status is set in this function (in cxa_demangle.cpp):

template

void

demangle(const char* first, const char* last, C& db, int& status)

{

if (first >= last)

{

status = invalid_mangled_name;

return;

}

if (*first == ‘_’)

{

if (last - first >= 4)

{

if (first[1] == ‘Z’)

{

const char* t = parse_encoding(first+2, last, db);

if (t != first+2 && t != last && *t == ‘.’)

t = parse_dot_suffix(t, last, db);

if (t != last)

status = invalid_mangled_name;

}

else if (first[1] == ‘’ && first[2] == '’ && first[3] == ‘Z’)

{

const char* t = parse_encoding(first+4, last, db);

if (t != first+4 && t != last)

{

const char* t1 = parse_block_invoke(t, last, db);

if (t1 != last)

status = invalid_mangled_name;

}

else

status = invalid_mangled_name;

}

else

status = invalid_mangled_name;

}

else

status = invalid_mangled_name;

}

else

{

const char* t = parse_type(first, last, db);

if (t != last)

—-> status = invalid_mangled_name; // here

}

if (status == success && db.names.empty())

status = invalid_mangled_name;

}

With these variable states:

(lldb) fr v

(const char *) first = 0x000000010006dce4 “Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”

(const char *) last = 0x000000010006ddbc “”

(__cxxabiv1::(anonymous namespace)::Db &) db = 0x00007fff5fbfe4c8: {

names = size=0 {}

subs = size=0 {}

template_param = size=1 {

[0] = size=0 {}

}

cv = 0

ref = 0

encoding_depth = 0

parsed_ctor_dtor_cv = false

tag_templates = true

fix_forward_references = false

try_to_parse_template_args = true

}

(int &) status = 0x00007fff5fbfe4b8 (&status = 0)

(const char *) t = 0x000000010006dce4 “Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”

Another symbol in question is this:

__ZZN5boost9function4IbRNSt3__111__wrap_iterIPKcEERKS5_RNS_6spirit7contextINS_6fusion4consIRN2fl3log8internal16enable_spec_elem13compound_maskENSB_4nil_EEENSB_7vector0IvEEEERKNS9_2qi10char_classINS9_3tag9char_codeINSR_5spaceENS9_13char_encoding5asciiEEEEEE9assign_toINSP_6detail13parser_binderINSP_4listINSP_11alternativeINSC_INSP_6actionINSP_9referenceINSP_7symbolsIcjNSP_3tstIcjEENSP_16tst_pass_throughEEEEENS_7phoenix5actorINS_5proto7exprns_10basic_exprINS1G_6tagns_3tag17bitwise_or_assignENS1G_7argsns_5list2INS1F_INS1I_INS1E_3tag4at_cENS1N_INS1I_INS1K_8terminalENS1M_4termIN4mpl_4int_ILi0EEEEELl0EEENS1F_INS9_9attributeILi0EEEEEEELl2EEEEENS1F_INS9_8argumentILi0EEEEEEELl2EEEEEEENSC_INS16_INSP_14literal_stringIRA5_S3_Lb1EEENS1F_INS1I_INS1K_6assignENS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IiEELl0EEEEEEELl2EEEEEEENSC_INS16_INS2A_IRA4_S3_Lb1EEENS1F_INS1I_IS2E_NS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IjEELl0EEEEEEELl2EEEEEEENSC_INS16_INS17_IKNSP_4ruleIS5_FjvENS1H_4exprIS1Q_NS1R_ISW_EELl0EEENS9_11unused_typeES31_EEEENS1F_INS1I_IS2E_NS1N_INS1I_IS1P_NS1N_INS1I_IS1Q_NS1R_INS1T_ILi1EEEEELl0EEES1Z_EELl2EEES25_EELl2EEEEEEESJ_EEEEEEEEEENSP_12literal_charINSU_8standardELb1ELb0EEEEENS1S_5bool_ILb0EEEEEEEvT_E13stored_vtable

Do you have suggestions on any other debugging techniques I could try here?

(also I plan to submit a bug report)

Thanks again,

-S

Re: [lldb-dev] Help with Xcode crash on lldb_private::(anonymous namespace)::parse_builtin_type

Jim Ingham via lldb-dev Mon, 29 Aug 2016 11:27:06 -0700

That is the demangler crashing on some C++ name in your program.

We added a "demangle" log channel to the lldb for Xcode 8.0 (it's also in TOT 
lldb) to make debugging this sort of crash easier.  Put:
log enable -f /tmp/lldb-demangle-log.txt lldb demangle

in your ~/.lldbinit file, then run your program till Xcode crashes, and look at 
the log file.  It will have a lot of output in pairs like:

demangle itanium: _Z12preoptimized
demangled itanium: _Z12preoptimized -> "preoptimized"

but then at the end will be an instance with the first line, not the second.  
That will be the problem. 

You can have multiple copies of Xcode on your system, so you wouldn't have to 
switch to 8.0 for your day-to-day work.  But if you can't easily switch to 
Xcode 8.0 for this analysis, you could try debugging Xcode with lldb, and 
setting a breakpoint at _cxa_demangle and print the incoming strings:

(lldb) expr (char *) $arg1

till you get the crash.  Do this in a breakpoint command with "continue" after 
the printing.  This is all optimized code so you are unlikely to be able to 
recover the full string that high up on the stack when you crash, you'll have 
to print all the strings at the call site.

LLDB uses a version of the system demangler code with some bug fixes.  It would 
be interesting to see if the system demangler also crashes on this:

$ c++filt <BAD_NAME>

But in any case, please file a bug with that bad symbol.  

Short of fixing the demangler bug, however, I can't see any easy way to work 
around this.  Our copy of the demangler hasn't changed since late May 2015, so 
this is more likely a latent bug tripped by some new symbol that is being 
generated by the compiler.  If you can identify  change in your code has 
started producing this troublesome symbol, you might be able to work around it 
that way, though this is not a great solution.

Jim

So any string that will be demangled by our internal demangler must start with "_Z". In the shared library or main application's executable mach-o file, the mangled names will contain an extra leading "_". We strip off this extra leading _ and will end up with a string of "_Z....". So your string of:

Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_

is not a mangled name. You might have seen this as part of a buffer that was in the middle of being demangled, but you will need to step out to the initial invocation of the demangle function to see the original string.

Your string of:

__ZZN5boost9function4IbRNSt3__111__wrap_iterIPKcEERKS5_RNS_6spirit7contextINS_6fusion4consIRN2fl3log8internal16enable_spec_elem13compound_maskENSB_4nil_EEENSB_7vector0IvEEEERKNS9_2qi10char_classINS9_3tag9char_codeINSR_5spaceENS9_13char_encoding5asciiEEEEEE9assign_toINSP_6detail13parser_binderINSP_4listINSP_11alternativeINSC_INSP_6actionINSP_9referenceINSP_7symbolsIcjNSP_3tstIcjEENSP_16tst_pass_throughEEEEENS_7phoenix5actorINS_5proto7exprns_10basic_exprINS1G_6tagns_3tag17bitwise_or_assignENS1G_7argsns_5list2INS1F_INS1I_INS1E_3tag4at_cENS1N_INS1I_INS1K_8terminalENS1M_4termIN4mpl_4int_ILi0EEEEELl0EEENS1F_INS9_9attributeILi0EEEEEEELl2EEEEENS1F_INS9_8argumentILi0EEEEEEELl2EEEEEEENSC_INS16_INSP_14literal_stringIRA5_S3_Lb1EEENS1F_INS1I_INS1K_6assignENS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IiEELl0EEEEEEELl2EEEEEEENSC_INS16_INS2A_IRA4_S3_Lb1EEENS1F_INS1I_IS2E_NS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IjEELl0EEEEEEELl2EEEEEEENSC_INS16_INS17_IKNSP_4ruleIS5_FjvENS1H_4exprIS1Q_NS1R_ISW_EELl0EEENS9_11unused_typeES31_EEEENS1F_INS1I_IS2E_NS1N_INS1I_IS1P_NS1N_INS1I_IS1Q_NS1R_INS1T_ILi1EEEEELl0EEES1Z_EELl2EEES25_EELl2EEEEEEESJ_EEEEEEEEEENSP_12literal_charINSU_8standardELb1ELb0EEEEENS1S_5bool_ILb0EEEEEEEvT_E13stored_vtable

Needs the leading "_" stripped before you try and demangle it with __cxxabiv1::__cxa_demangle() or it will fail. The extra leading "_" is a mach-o C symbol namespace thing that doesn't belong in the name. c++filt will automatically strip it unless you specify -n or --no-strip-underscore.

That being said, you can just enable the logging in lldb and run it and it will print out what it is trying to demangle before it gets demangled and you can just let LLDB crash and you should be able to see the string in the STDOUT of the program you are debugging.

If you have any of your problem binaries that you can share, just attach them to the bug report and we can easily figure out what is causing the problems by demangling all strings inside of them.