Hi Jim,
Thank you for the response!! I hadn’t subscribed to the lldb-dev mailing list until now, so I did not receive your response until googling for it 
Anyways, between when I sent my email and seeing your response, I tried very similar things to what you mentioned. I built lldb from latest source and replaced LLDB.framework in one of my Xcode versions with the version I built, adding some of my own logging, and also turned on demangle logging (I now know that I could have just used Xcode 8).
I am seeing some interesting results. To give a bit more background on the problem, let me explain it in a simplified version. Imagine we have 200 unit test executables all with their own main(), cross-compiled for iOS arm64. I put these into generated Xcode projects, and we use the Xcode projects to “build and run”. This allows Xcode to launch an executable as an app on an iOS device. Now imagine that each executable is a unit test for an iOS static library, so in this scenario we have 200 static libraries. Let’s say they are all in a DAG, so library 200 depends on library 199 and library 199 depends on library 198, etc. About halfway through the DAG, e.g. library 100, the demangler starts to crash. In this example, every unit test executable downstream of library 100 and including library 100 are causing the demangler to crash. It seems out best bet here is to find the offending symbol(s) and attempt to replace the code in our codebase with something the demangler can handle.
We have run the demangle log with several different executables and the demangler seems to crash on similar but slightly different symbols each time (of course, the symbol the demangler appears to be crashing on isn’t necessarily the offending symbol).
Using lldb from Xcode 7.3.1, I set Xcode 7.3.0 as my target executable and reproduced the crash, which allowed me to see the EXC_BAD_ACCESS in the debugger (and since I have Xcode 7.3.0 using my locally built version of lldb, I get to see lldb source information :p) :
Process 30313 stopped
- thread #6: tid = 0xe0d1a, 0x00000001169d19b1 LLDB`char const* lldb_private::(anonymous namespace)::parse_expr_primary<lldb_private::(first=“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”, last=“”, db=0x0000700000216a58)::Db>(char const*, char const*, lldb_private::(anonymous namespace)::Db&) + 97 at CxaDemangle.cpp:2690, queue = ‘DBGLLDBLauncher Serial Queue’, stop reason = EXC_BAD_ACCESS (code=2, address=0x70000019cd98)
frame #0: 0x00000001169d19b1 LLDB`char const* lldb_private::(anonymous namespace)::parse_expr_primary<lldb_private::(first=“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”, last=“”, db=0x0000700000216a58)::Db>(char const*, char const*, lldb_private::(anonymous namespace)::Db&) + 97 at CxaDemangle.cpp:2690
2687 {
2688 if (last - first >= 4 && *first == ‘L’)
2689 {
→ 2690 switch (first[1])
2691 {
2692 case ‘w’:
2693 {
(lldb) quit
This the symbol from the above crash:
Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_
If I run c++filt, it just returns the mangled name back to me:
[sburke-maci:/Users/sburke]2 % c++filt Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_
Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_
Something similar happens if I run __cxa_demangle directly against the symbols we are investigating, including the one above. The demangler does not crash but the symbol is considered an “invalid mangled name” and cannot be demangled.
e.g.:
#include
#include “cxxabi.h"
int main(int argc, const char * argv) {
const char *bad_symbol1 =
“__ZZN5boost9function4IbRNSt3__111__wrap_iterIPKcEERKS5_RNS_6spirit7contextINS_6fusion4consIRN2fl3log8internal16enable_spec_elem13compound_maskENSB_4nil_EEENSB_7vector0IvEEEERKNS9_2qi10char_classINS9_3tag9char_codeINSR_5spaceENS9_13char_encoding5asciiEEEEEE9assign_toINSP_6detail13parser_binderINSP_4listINSP_11alternativeINSC_INSP_6actionINSP_9referenceINSP_7symbolsIcjNSP_3tstIcjEENSP_16tst_pass_throughEEEEENS_7phoenix5actorINS_5proto7exprns_10basic_exprINS1G_6tagns_3tag17bitwise_or_assignENS1G_7argsns_5list2INS1F_INS1I_INS1E_3tag4at_cENS1N_INS1I_INS1K_8terminalENS1M_4termIN4mpl_4int_ILi0EEEEELl0EEENS1F_INS9_9attributeILi0EEEEEEELl2EEEEENS1F_INS9_8argumentILi0EEEEEEELl2EEEEEEENSC_INS16_INSP_14literal_stringIRA5_S3_Lb1EEENS1F_INS1I_INS1K_6assignENS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IiEELl0EEEEEEELl2EEEEEEENSC_INS16_INS2A_IRA4_S3_Lb1EEENS1F_INS1I_IS2E_NS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IjEELl0EEEEEEELl2EEEEEEENSC_INS16_INS17_IKNSP_4ruleIS5_FjvENS1H_4exprIS1Q_NS1R_ISW_EELl0EEENS9_11unused_typeES31_EEEENS1F_INS1I_IS2E_NS1N_INS1I_IS1P_NS1N_INS1I_IS1Q_NS1R_INS1T_ILi1EEEEELl0EEES1Z_EELl2EEES25_EELl2EEEEEEESJ_EEEEEEEEEENSP_12literal_charINSU_8standardELb1ELb0EEEEENS1S_5bool_ILb0EEEEEEEvT_E13stored_vtable”;
const char *bad_symbol2 =
“Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”;
size_t length = 0;
int status = 0;
//0: The demangling operation succeeded.
//-1: A memory allocation failiure occurred.
//-2: mangled_name is not a valid name under the C++ ABI mangling rules.
//-3: One of the arguments is invalid.
char *demangleChars = __cxxabiv1::__cxa_demangle(bad_symbol2, nullptr, &length, &status);
std::cout << "status = " << status << std::endl;
std::cout << "length = " << length << std::endl;
if(demangleChars) {
std::cout << demangleChars << std::endl;
free(demangleChars);
}
return 0;
}
This gives:
status = -2
length = 0
The status is set in this function (in cxa_demangle.cpp):
template
void
demangle(const char* first, const char* last, C& db, int& status)
{
if (first >= last)
{
status = invalid_mangled_name;
return;
}
if (*first == ‘_’)
{
if (last - first >= 4)
{
if (first[1] == ‘Z’)
{
const char* t = parse_encoding(first+2, last, db);
if (t != first+2 && t != last && *t == ‘.’)
t = parse_dot_suffix(t, last, db);
if (t != last)
status = invalid_mangled_name;
}
else if (first[1] == ‘’ && first[2] == '’ && first[3] == ‘Z’)
{
const char* t = parse_encoding(first+4, last, db);
if (t != first+4 && t != last)
{
const char* t1 = parse_block_invoke(t, last, db);
if (t1 != last)
status = invalid_mangled_name;
}
else
status = invalid_mangled_name;
}
else
status = invalid_mangled_name;
}
else
status = invalid_mangled_name;
}
else
{
const char* t = parse_type(first, last, db);
if (t != last)
—-> status = invalid_mangled_name; // here
}
if (status == success && db.names.empty())
status = invalid_mangled_name;
}
With these variable states:
(lldb) fr v
(const char *) first = 0x000000010006dce4 “Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”
(const char *) last = 0x000000010006ddbc “”
(__cxxabiv1::(anonymous namespace)::Db &) db = 0x00007fff5fbfe4c8: {
names = size=0 {}
subs = size=0 {}
template_param = size=1 {
[0] = size=0 {}
}
cv = 0
ref = 0
encoding_depth = 0
parsed_ctor_dtor_cv = false
tag_templates = true
fix_forward_references = false
try_to_parse_template_args = true
}
(int &) status = 0x00007fff5fbfe4b8 (&status = 0)
(const char *) t = 0x000000010006dce4 “Ll2EEEEENS1H_2idINS1H_6threadEEES13_S15_EEEES1B_EELl2EEEEENSN_12basic_stringIcS1L_S1N_EES13_S15_EEEES1B_EELl2EEEEEN2fl3log19logging_event_typesES13_S15_EEEES1B_EELl2EEEEENS2F_8internal16indent_formatterIcEEEC2ERKS2P_”
Another symbol in question is this:
__ZZN5boost9function4IbRNSt3__111__wrap_iterIPKcEERKS5_RNS_6spirit7contextINS_6fusion4consIRN2fl3log8internal16enable_spec_elem13compound_maskENSB_4nil_EEENSB_7vector0IvEEEERKNS9_2qi10char_classINS9_3tag9char_codeINSR_5spaceENS9_13char_encoding5asciiEEEEEE9assign_toINSP_6detail13parser_binderINSP_4listINSP_11alternativeINSC_INSP_6actionINSP_9referenceINSP_7symbolsIcjNSP_3tstIcjEENSP_16tst_pass_throughEEEEENS_7phoenix5actorINS_5proto7exprns_10basic_exprINS1G_6tagns_3tag17bitwise_or_assignENS1G_7argsns_5list2INS1F_INS1I_INS1E_3tag4at_cENS1N_INS1I_INS1K_8terminalENS1M_4termIN4mpl_4int_ILi0EEEEELl0EEENS1F_INS9_9attributeILi0EEEEEEELl2EEEEENS1F_INS9_8argumentILi0EEEEEEELl2EEEEEEENSC_INS16_INSP_14literal_stringIRA5_S3_Lb1EEENS1F_INS1I_INS1K_6assignENS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IiEELl0EEEEEEELl2EEEEEEENSC_INS16_INS2A_IRA4_S3_Lb1EEENS1F_INS1I_IS2E_NS1N_IS21_NS1F_INS1I_IS1Q_NS1R_IjEELl0EEEEEEELl2EEEEEEENSC_INS16_INS17_IKNSP_4ruleIS5_FjvENS1H_4exprIS1Q_NS1R_ISW_EELl0EEENS9_11unused_typeES31_EEEENS1F_INS1I_IS2E_NS1N_INS1I_IS1P_NS1N_INS1I_IS1Q_NS1R_INS1T_ILi1EEEEELl0EEES1Z_EELl2EEES25_EELl2EEEEEEESJ_EEEEEEEEEENSP_12literal_charINSU_8standardELb1ELb0EEEEENS1S_5bool_ILb0EEEEEEEvT_E13stored_vtable
Do you have suggestions on any other debugging techniques I could try here?
(also I plan to submit a bug report)
Thanks again,
-S
Re: [lldb-dev] Help with Xcode crash on lldb_private::(anonymous namespace)::parse_builtin_type
Jim Ingham via lldb-dev Mon, 29 Aug 2016 11:27:06 -0700
That is the demangler crashing on some C++ name in your program.
We added a "demangle" log channel to the lldb for Xcode 8.0 (it's also in TOT
lldb) to make debugging this sort of crash easier. Put:
log enable -f /tmp/lldb-demangle-log.txt lldb demangle
in your ~/.lldbinit file, then run your program till Xcode crashes, and look at
the log file. It will have a lot of output in pairs like:
demangle itanium: _Z12preoptimized
demangled itanium: _Z12preoptimized -> "preoptimized"
but then at the end will be an instance with the first line, not the second.
That will be the problem.
You can have multiple copies of Xcode on your system, so you wouldn't have to
switch to 8.0 for your day-to-day work. But if you can't easily switch to
Xcode 8.0 for this analysis, you could try debugging Xcode with lldb, and
setting a breakpoint at _cxa_demangle and print the incoming strings:
(lldb) expr (char *) $arg1
till you get the crash. Do this in a breakpoint command with "continue" after
the printing. This is all optimized code so you are unlikely to be able to
recover the full string that high up on the stack when you crash, you'll have
to print all the strings at the call site.
LLDB uses a version of the system demangler code with some bug fixes. It would
be interesting to see if the system demangler also crashes on this:
$ c++filt <BAD_NAME>
But in any case, please file a bug with that bad symbol.
Short of fixing the demangler bug, however, I can't see any easy way to work
around this. Our copy of the demangler hasn't changed since late May 2015, so
this is more likely a latent bug tripped by some new symbol that is being
generated by the compiler. If you can identify change in your code has
started producing this troublesome symbol, you might be able to work around it
that way, though this is not a great solution.
Jim