lld/lto/win32 crash on DIE code

I have a fairly recent LLD/LTO llvm crashing on

DIE *ContextDIE = getOrCreateContextDIE(Context)
being null for a (local) variable. (Context is a DICompileUnit in this case, but it's not present in MDNodeToDieMap so it returns null. callstack is:

llc.exe!llvm::DwarfUnit::getOrCreateTypeDIE(const llvm::MDNode * TyNode) Line 718 C++
llvm::DwarfUnit::addType(llvm::DIE & Entity, const llvm::DIType * Ty, llvm::dwarf::Attribute Attribute) Line 768 C++
llvm::DwarfCompileUnit::applyVariableAttributes(const llvm::DbgVariable & Var, llvm::DIE & VariableDie) Line 897 C++
llvm::DwarfCompileUnit::finishVariableDefinition(const llvm::DbgVariable & Var) Line 725 C++
llvm::DwarfDebug::finishVariableDefinitions() Line 655 C++
llvm::DwarfDebug::finalizeModuleInfo() Line 677 C++
llvm::DwarfDebug::endModule() Line 768 C++
llvm::AsmPrinter::doFinalization(llvm::Module & M) Line 1348 C++
llvm::FPPassManager::doFinalization(llvm::Module & M) Line 1559 C++
`anonymous namespace'::MPPassManager::runOnModule(llvm::Module & M) Line 1615 C++
llvm::legacy::PassManagerImpl::run(llvm::Module & M) Line 1700 C++
llvm::legacy::PassManager::run(llvm::Module & M) Line 1732 C++
compileModule(char * * argv, llvm::LLVMContext & Context) Line 572 C++

The input of this testcase is huge, full repro.tar is here: (14mb)
https://www.dropbox.com/s/ugv9sfqouhlszff/repro.tar?dl=0

With lld /lldsavetemps I managed to narrow down what it actually fails on, specifcally from the Island.Tests.Windows.exe.0.5.precodegen.bc (which fails too when passed to llc -O1)

This is the pattern that it generates:
https://gist.github.com/carlokok/fb0f1bf213ee4599d8f2442e37b23f03

If I read this correctly it somehow ends up using two compile units at once from the same type and failing because of that. I don't know if that's a merging bug or something I'm doing, but none of my input bc files has more than 1 compileunit per file, so I don't know how this could have happened.

What can I do to solve this?

Thanks,

carlo Kok

Hello Carlo,

I tried your reproducer and faced different problem from one you described
(I'm using MacOS Sierra and lld built from trunk on Mar, 15). The crash happens
when SelectionDAGBuilder::lowerInvokable tries to access EH info of this function:

ms_t26_RemObjects_d_Elements_d_EUnit_d_Runnerb_RunChildrennt2a_RemObjects_d_Elements_d_EUnit_d_RunContext

This happens because LLVM doesn't know your personality function (@_elements_exception_handler), so I suggest looking
at llvm::classifyEHPersonality function as well.

Ah damn yes. I have a modified llvm to support my custom personality. I’ll try and prepare one with standard personalities Monday.

hi,

I've updated my testcase
https://www.dropbox.com/s/ugv9sfqouhlszff/repro.tar?dl=0

to use the standard exception handler. Now it fails with llvm-6.0 from the website exactly the same as it does with my custom build.

This one triggers an assertion in calculateSEHStateNumbers due to weird catchpad instruction
in @_island_debug_invoke and many other functions. The code expects either pointer to a filter
function or null in first operand, while you're passing pointer to structure:

catchpad within %80 [{i8*, i8*}* anon..., ...]

This one triggers an assertion in calculateSEHStateNumbers due to weird catchpad instruction
in @_island_debug_invoke and many other functions. The code expects either pointer to a filter
function or null in first operand, while you're passing pointer to structure:

catchpad within %80 [{i8*, i8*}* anon..., ...]

________________________________________

I used the wrong exception method name, using the regular one now (__CxxFrameHandler3). Also tested it in latest clean llvm/debug and crashes how I get it locally too.

https://www.dropbox.com/s/n3e2eystps9qvwx/repro.tar?dl=0

> lld.exe!HandleAbort(int Sig) Line 411 C++ Symbols loaded.
    ucrtbased.dll!raise(int signum) Line 541 C++ Symbols loaded.
    ucrtbased.dll!abort() Line 64 C++ Symbols loaded.
    ucrtbased.dll!common_assert_to_stderr_direct(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 161 C++ Symbols loaded.
    ucrtbased.dll!common_assert_to_stderr<wchar_t>(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number) Line 175 C++ Symbols loaded.
    ucrtbased.dll!common_assert<wchar_t>(const wchar_t * const expression, const wchar_t * const file_name, const unsigned int line_number, void * const return_address) Line 415 C++ Symbols loaded.
    ucrtbased.dll!_wassert(const wchar_t * expression, const wchar_t * file_name, unsigned int line_number) Line 443 C++ Symbols loaded.
    lld.exe!llvm::DwarfUnit::getOrCreateTypeDIE(const llvm::MDNode * TyNode) Line 753 C++ Symbols loaded.
    lld.exe!llvm::DwarfUnit::addType(llvm::DIE & Entity, const llvm::DIType * Ty, llvm::dwarf::Attribute Attribute) Line 803 C++ Symbols loaded.
    lld.exe!llvm::DwarfCompileUnit::applyVariableAttributes(const llvm::DbgVariable & Var, llvm::DIE & VariableDie) Line 980 C++ Symbols loaded.
    lld.exe!llvm::DwarfCompileUnit::finishVariableDefinition(const llvm::DbgVariable & Var) Line 808 C++ Symbols loaded.
    lld.exe!llvm::DwarfDebug::finishVariableDefinitions() Line 671 C++ Symbols loaded.
    lld.exe!llvm::DwarfDebug::finalizeModuleInfo() Line 693 C++ Symbols loaded.
    lld.exe!llvm::DwarfDebug::endModule() Line 784 C++ Symbols loaded.
    lld.exe!llvm::AsmPrinter::doFinalization(llvm::Module & M) Line 1352 C++ Symbols loaded.
    lld.exe!llvm::FPPassManager::doFinalization(llvm::Module & M) Line 1559 C++ Symbols loaded.
    lld.exe!`anonymous namespace'::MPPassManager::runOnModule(llvm::Module & M) Line 1615 C++ Symbols loaded.
    lld.exe!llvm::legacy::PassManagerImpl::run(llvm::Module & M) Line 1700 C++ Symbols loaded.
    lld.exe!llvm::legacy::PassManager::run(llvm::Module & M) Line 1732 C++ Symbols loaded.
    lld.exe!`anonymous namespace'::codegen(llvm::lto::Config & Conf, llvm::TargetMachine * TM, std::function<std::unique_ptr<llvm::lto::NativeObjectStream,std::default_delete<llvm::lto::NativeObjectStream> > __cdecl(unsigned int)> AddStream, unsigned int Task, llvm::Module & Mod) Line 292 C++ Symbols loaded.
    lld.exe!llvm::lto::backend(llvm::lto::Config & C, std::function<std::unique_ptr<llvm::lto::NativeObjectStream,std::default_delete<llvm::lto::NativeObjectStream> > __cdecl(unsigned int)> AddStream, unsigned int ParallelCodeGenParallelismLevel, std::unique_ptr<llvm::Module,std::default_delete<llvm::Module> > Mod, llvm::ModuleSummaryIndex & CombinedIndex) Line 393 C++ Symbols loaded.
lld.exe!llvm::lto::LTO::runRegularLTO(std::function<std::unique_ptr<llvm::lto::NativeObjectStream,std::default_delete<llvm::lto::NativeObjectStream> > __cdecl(unsigned int)> AddStream) Line 860 C++ Symbols loaded.
lld.exe!llvm::lto::LTO::run(std::function<std::unique_ptr<llvm::lto::NativeObjectStream,std::default_delete<llvm::lto::NativeObjectStream> > __cdecl(unsigned int)> AddStream, std::function<std::function<std::unique_ptr<llvm::lto::NativeObjectStream,std::default_delete<llvm::lto::NativeObjectStream> > __cdecl(unsigned int)> __cdecl(unsigned int,llvm::StringRef)> Cache) Line 793 C++ Symbols loaded.
    lld.exe!lld::coff::BitcodeCompiler::compile() Line 141 C++ Symbols loaded.
    lld.exe!lld::coff::SymbolTable::compileBitcodeFiles() Line 381 C++ Symbols loaded.
    lld.exe!lld::coff::SymbolTable::addCombinedLTOObjects() Line 389 C++ Symbols loaded.
    lld.exe!lld::coff::LinkerDriver::link(llvm::ArrayRef<char const *> ArgsArr) Line 1366 C++ Symbols loaded.
    lld.exe!lld::coff::link(llvm::ArrayRef<char const *> Args, bool CanExitEarly, llvm::raw_ostream & Diag) Line 75 C++ Symbols loaded.
    lld.exe!main(int Argc, const char * * Argv) Line 129 C++ Symbols loaded.
    lld.exe!invoke_main() Line 78 C++ Symbols loaded.
    lld.exe!__scrt_common_main_seh() Line 283 C++ Symbols loaded.
    lld.exe!__scrt_common_main() Line 326 C++ Symbols loaded.
    lld.exe!mainCRTStartup() Line 17 C++ Symbols loaded.
    kernel32.dll!75568654() Unknown No symbols loaded.
    [Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll] Annotated Frame
    ntdll.dll!__RtlUserThreadStart() Unknown Symbols loaded.
    ntdll.dll!__RtlUserThreadStart@8() Unknown Symbols loaded.

It looks the problem lies in how your compiler generates debug info. LLVM doesn't
expect DIDerivedType scope to be an instance of DICompileUnit. Here is a quick fix:

DIE *DwarfUnit::getOrCreateContextDIE(const DIScope *Context) {
- if (!Context || isa<DIFile>(Context))
+ if (!Context || isa<DIFile>(Context) || isa<DICompileUnit>(Context))

However, I suggest talking to someone with in-depth debug info experience, as the problem
itself seems to be not related to LTO.

The problem seems to be related to that but it *does* support some DICompileUnits, just not this particular one, I stepped through this code and it gets a DICompileUnit often enough. Somehow during LTO/merging I get a type that has a different DICompileUnit than where the variable is defined, and it's not liking it.

Ok, I've done a bit more investigation. I found the module which declares broken DIE
(e54b3dc8c0536e29a65f8548b5ae7958-Global.o) and here is what I found there:

!2 = !DIFile(filename: "island.windows.elements", ...
...
!4 = distinct !DICompileUnit(language: DW_LANG_C99, file: !5, ...
!5 = !DIFile(filename: "island.windows.elements-e54b3dc8c0536e29a65f8548b5ae7958-global", ...

All subsequent DIEs reference file !2, except DICompileUnit (!4) which references file !5. This seems a bit
strange for me. Can't this be a root cause of your problems?

Thanks!

Unfortunately this doesn't seem to cause it, because when I fix it to match the other files (and pretty much how clang emits it:)

!0 = !DIGlobalVariableExpression(var: !1, expr: !DIExpression())
!1 = !DIGlobalVariable(name: "IDispatch_UID", linkageName: "f_t2b_RemObjects_d_Elements_d_System_d_____Global.IDispatchUID", scope: !2, file: !3, type: !622, isLocal: false, isDefinition: true)
!2 = distinct !DICompileUnit(language: DW_LANG_C99, file: !3, producer: "RemObjects Island", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !4, retainedTypes: !4, globals: !5, imports: !703)
!3 = !DIFile(filename: "island.windows.elements-e54b3dc8c0536e29a65f8548b5ae7958-global", directory: "/__windows_drive__c/ci/b/elements/1301/source/islandrtl/source")
!4 = !{}
!5 = !{!0, !6, !620, !635, !637, !639, !660, !669, !671, !673, !675, !680, !682, !684}
!6 = !DIGlobalVariableExpression(var: !7, expr: !DIExpression())
!7 = !DIGlobalVariable(name: "IDispatch_VMT", linkageName: "f_t2b_RemObjects_d_Elements_d_System_d_____Global.IDispatchVMT", scope: !2, file: !3, type: !8, isLocal: false, isDefinition: true)

It still crashes. Same callstack.

Note that all these .o files individually compile file, it's only when I let the linker lto-merge them things go wrong for me.

(updated https://www.dropbox.com/s/n3e2eystps9qvwx/repro.tar?dl=0 to reflect above)

Yep, it's once again variable and it's type using different scopes.
Is there any strong reason why you set scope for global DIDerivedType?

I thought I had to do that. I’ll try without scope. Thanks!

Evgeny Leviant eleviant@accesssoftek.com schreef op 21 maart 2018 18:36:26 CET:

So I removed them all and everything works now. I'm still curious why it didn't work but since everything compiles now, so I can move on. Thank you!