Trouble with clang::SourceManager and clang::ASTUnit

Hi all,

I'm writing an auto-completion plug-in for Sublime Text 3 with a compiled component in C++. The compiled component is written as a clang tool living in llvm/tools/clang/tools/ and it links with the clangFrontend library. The bridge between Sublime's Python interface and C++ is pybind11.
So, for every suitable open buffer in Sublime I have a clang::ASTUnit fetching completions for it. The preamble of each file definitely needs to be precompiled otherwise fetching completions is too slow for serious use.

When I construct a clang::ASTUnit via ASTUnit::LoadFromCompilerInvocation (I build the invocation myself), and I choose the parameter PrecompilePreambleAfterNParses to be 1, then I sometimes get an assertion error involving clang::SourceManager, saying that two SourceLocations don't come from the same SourceManager.

I don't know how to fix this problem, as I cannot pass my own SourceManager anywhere. And it happens during construction of an ASTUnit, so I cannot re-arrange calls anywhere. If anyone has an idea as to what causes this then I'd be very interested, because I can't continue developing this plugin unless I fix this issue.

I have found out that if I set PrecompilePreambleAfterNParses to 0, everything seems to work fine. So, I'm guessing it has something to do with precompiling the preamble somehow. The full stack trace of the assertion error is below. If someone needs more info like local variables I can provide that too.

Regards,

Raoul Wols

* thread #6: tid = 0x6aed, 0x00007fffbeb0edda libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
   * frame #0: 0x00007fffbeb0edda libsystem_kernel.dylib`__pthread_kill + 10
     frame #1: 0x00007fffbebfa787 libsystem_pthread.dylib`pthread_kill + 90
     frame #2: 0x00007fffbea74420 libsystem_c.dylib`abort + 129
     frame #3: 0x00007fffbea3b893 libsystem_c.dylib`__assert_rtn + 320
     frame #4: 0x000000010e98b63e Clara.so`clang::FullSourceLoc::isBeforeInTranslationUnitThan(this=0x000070000406f3f8, Loc=FullSourceLoc @ 0x000070000406f240) const + 174 at SourceLocation.h:319
     frame #5: 0x000000010e997cec Clara.so`clang::DiagnosticsEngine::DiagStatePoint::operator<(clang::DiagnosticsEngine::DiagStatePoint const&) const(this=0x000070000406f3f0, RHS=0x00007f886218a2f0) + 140 at Diagnostic.h:245
     frame #6: 0x000000010e997bd4 Clara.so`std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*> std::__1::__upper_bound<std::__1::__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint>&, std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, clang::DiagnosticsEngine::DiagStatePoint>(std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, clang::DiagnosticsEngine::DiagStatePoint const&, std::__1::__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint>&) [inlined] std::__1::__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint>::operator(this=0x000070000406f4e8, __x=0x000070000406f3f0, __y=0x00007f886218a2f0)(clang::DiagnosticsEngine::DiagStatePoint const&, clang::DiagnosticsEngine::DiagStatePoint const&) const + 436 at algorithm:702
     frame #7: 0x000000010e997bc7 Clara.so`std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*> std::__1::__upper_bound<std::__1::__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint>&, std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, clang::DiagnosticsEngine::DiagStatePoint>(__first=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f2f8, __last=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f2f0, __value_=0x000070000406f3f0, __comp=0x000070000406f4e8) + 423 at algorithm:4180
     frame #8: 0x000000010e98b51d Clara.so`clang::DiagnosticsEngine::GetDiagStatePointForLoc(clang::SourceLocation) const [inlined] std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*> std::__1::upper_bound<std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, clang::DiagnosticsEngine::DiagStatePoint, std::__1::__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint> >(__first=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f4f8, __last=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f4f0, __value_=0x000070000406f3f0, __comp=__less<clang::DiagnosticsEngine::DiagStatePoint, clang::DiagnosticsEngine::DiagStatePoint> @ 0x000070000406f4e8) + 54 at algorithm:4202
     frame #9: 0x000000010e98b4e7 Clara.so`clang::DiagnosticsEngine::GetDiagStatePointForLoc(clang::SourceLocation) const [inlined] std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*> std::__1::upper_bound<std::__1::__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint*>, clang::DiagnosticsEngine::DiagStatePoint>(__first=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f4c0, __last=__wrap_iter<clang::DiagnosticsEngine::DiagStatePoint *> @ 0x000070000406f4b8, __value_=0x000070000406f3f0) + 70 at algorithm:4211
     frame #10: 0x000000010e98b4a1 Clara.so`clang::DiagnosticsEngine::GetDiagStatePointForLoc(this=0x00007f8861189e00, L=(ID = 825)) const + 1537 at Diagnostic.cpp:177
     frame #11: 0x000000010e99cb6e Clara.so`clang::DiagnosticIDs::getDiagnosticSeverity(this=0x00007f886009f390, DiagID=930, Loc=(ID = 825), Diag=0x00007f8861189e00) const + 126 at DiagnosticIDs.cpp:415
     frame #12: 0x000000010e904921 Clara.so`clang::DiagnosticsEngine::isIgnored(this=0x00007f8861189e00, DiagID=930, Loc=(ID = 825)) const + 65 at Diagnostic.h:652
     frame #13: 0x000000010e8e32aa Clara.so`clang::Preprocessor::HandleDefineDirective(this=0x00007f886367e200, DefineTok=0x00007f8862810610, ImmediatelyAfterHeaderGuard=false) + 4170 at PPDirectives.cpp:2546
     frame #14: 0x000000010e8daf9c Clara.so`clang::Preprocessor::HandleDirective(this=0x00007f886367e200, Result=0x00007f8862810610) + 1756 at PPDirectives.cpp:1013
     frame #15: 0x000000010e88316a Clara.so`clang::Lexer::LexTokenInternal(this=0x00007f886218c8c0, Result=0x00007f8862810610, TokAtPhysicalStartOfLine=true) + 9370 at Lexer.cpp:3681
     frame #16: 0x000000010e87eb78 Clara.so`clang::Lexer::Lex(this=0x00007f886218c8c0, Result=0x00007f8862810610) + 216 at Lexer.cpp:2931
     frame #17: 0x000000010e95e29e Clara.so`clang::Preprocessor::Lex(this=0x00007f886367e200, Result=0x00007f8862810610) + 126 at Preprocessor.cpp:745
     frame #18: 0x000000010d057429 Clara.so`clang::Parser::ConsumeToken(this=0x00007f8862810600) + 153 at Parser.h:307
     frame #19: 0x000000010d05dd58 Clara.so`clang::Parser::Initialize(this=0x00007f8862810600) + 5400 at Parser.cpp:530
     frame #20: 0x000000010cf3be38 Clara.so`clang::ParseAST(S=0x00007f886280d800, PrintStats=false, SkipFunctionBodies=false) + 696 at ParseAST.cpp:139
     frame #21: 0x000000010ce5ef35 Clara.so`clang::ASTFrontendAction::ExecuteAction(this=0x00007f886206d360) + 485 at FrontendAction.cpp:557
     frame #22: 0x000000010ce5d96f Clara.so`clang::FrontendAction::Execute(this=0x00007f886206d360) + 143 at FrontendAction.cpp:458
     frame #23: 0x000000010cd20c97 Clara.so`clang::ASTUnit::Parse(this=0x00007f8861233000, PCHContainerOps=nullptr, OverrideMainBuffer=unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer> > @ 0x0000700004071f78) + 8503 at ASTUnit.cpp:1145
     frame #24: 0x000000010cd35acf Clara.so`clang::ASTUnit::LoadFromCompilerInvocation(this=0x00007f8861233000, PCHContainerOps=nullptr, PrecompilePreambleAfterNParses=1) + 2015 at ASTUnit.cpp:1900
     frame #25: 0x000000010cd3655b Clara.so`clang::ASTUnit::LoadFromCompilerInvocation(CI=0x00007f88608bc400, PCHContainerOps=nullptr, Diags=IntrusiveRefCntPtr<clang::DiagnosticsEngine> @ 0x0000700004072b60, FileMgr=0x00007f8862184120, OnlyLocalDecls=false, CaptureDiagnostics=false, PrecompilePreambleAfterNParses=1, TUKind=TU_Complete, CacheCodeCompletionResults=true, IncludeBriefCommentsInCodeCompletion=true, UserFilesAreVolatile=true) + 1483 at ASTUnit.cpp:1933
     frame #26: 0x000000010cc4b960 Clara.so`Clara::Session::Session(this=0x00007f886009ee00, options=0x00007f886214dc80) + 1936 at Session.cpp:23
     frame #27: 0x000000010cc4ccad Clara.so`Clara::Session::Session(this=0x00007f886009ee00, options=0x00007f886214dc80) + 29 at Session.cpp:15
     frame #28: 0x000000010cba07e4 Clara.so`void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(this=0x00007f8862061268, self_=0x00007f886009ee00, args=0x00007f886214dc80)::'lambda'(Clara::Session*, Clara::SessionOptions const&)::operator()(Clara::Session*, Clara::SessionOptions const&) const + 36 at pybind11.h:1282
     frame #29: 0x000000010cba07b9 Clara.so`pybind11::class_<Clara::Session> pybind11::detail::type_caster<std::__1::tuple<Clara::Session*, Clara::SessionOptions const&>, void>::call<void, void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(this=0x0000700004073080, f=0x00007f8862061268, (null)=index_sequence<0, 1> @ 0x0000700004072f88)::'lambda'(Clara::Session*, Clara::SessionOptions const&)&, 0ul, 1ul>(void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(pybind11::class_<Clara::Session>&)::'lambda'(Clara::Session*, Clara::SessionOptions const&)&&&, pybind11::detail::index_sequence<0ul, 1ul>) + 105 at cast.h:834
     frame #30: 0x000000010cba0405 Clara.so`std::__1::enable_if<std::is_void<pybind11::class_<Clara::Session> >::value, pybind11::detail::void_type>::type pybind11::detail::type_caster<std::__1::tuple<Clara::Session*, Clara::SessionOptions const&>, void>::call<void, void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(this=0x0000700004073080, f=0x00007f8862061268)::'lambda'(Clara::Session*, Clara::SessionOptions const&)&>(void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(pybind11::class_<Clara::Session>&)::'lambda'(Clara::Session*, Clara::SessionOptions const&)&&&) + 37 at cast.h:822
     frame #31: 0x000000010cba02d7 Clara.so`void pybind11::cpp_function::initialize<void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(pybind11::class_<Clara::Session>&)::'lambda'(Clara::Session*, Clara::SessionOptions const&), void, Clara::Session*, Clara::SessionOptions const&, pybind11::name, pybind11::is_method, pybind11::sibling>(this=0x000000010cba0190, rec=0x00007f8862061230, args=handle @ 0x00007000040730d0, kwargs=handle @ 0x00007000040730c8, parent=handle @ 0x00007000040730c0)(Clara::Session*, Clara::SessionOptions const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)::operator()(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle) const + 247 at pybind11.h:140
     frame #32: 0x000000010cba01c0 Clara.so`void pybind11::cpp_function::initialize<void pybind11::detail::init<Clara::SessionOptions const&>::execute<pybind11::class_<Clara::Session>, 0>(pybind11::class_<Clara::Session>&)::'lambda'(Clara::Session*, Clara::SessionOptions const&), void, Clara::Session*, Clara::SessionOptions const&, pybind11::name, pybind11::is_method, pybind11::sibling>(rec=0x00007f8862061230, args=handle @ 0x0000700004073110, kwargs=handle @ 0x0000700004073108, parent=handle @ 0x0000700004073100)(Clara::Session*, Clara::SessionOptions const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&)::'lambda'(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle)::__invoke(pybind11::detail::function_record*, pybind11::handle, pybind11::handle, pybind11::handle) + 48 at pybind11.h:120
     frame #33: 0x000000010cb8653e Clara.so`pybind11::cpp_function::dispatcher(self=0x000000010c4f1330, args=0x000000010c84ff80, kwargs=0x0000000000000000) + 2014 at pybind11.h:436
     frame #34: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #35: 0x000000010a59f5c9 plugin_host`method_call + 136
     frame #36: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #37: 0x000000010a5d3186 plugin_host`slot_tp_init + 57
     frame #38: 0x000000010a5d0726 plugin_host`type_call + 172
     frame #39: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #40: 0x000000010a621a28 plugin_host`PyEval_EvalFrameEx + 20121
     frame #41: 0x000000010a61c9d8 plugin_host`PyEval_EvalCodeEx + 1622
     frame #42: 0x000000010a5ad8f1 plugin_host`function_call + 365
     frame #43: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #44: 0x000000010a622226 plugin_host`PyEval_EvalFrameEx + 22167
     frame #45: 0x000000010a625820 plugin_host`fast_function + 203
     frame #46: 0x000000010a621afe plugin_host`PyEval_EvalFrameEx + 20335
     frame #47: 0x000000010a625820 plugin_host`fast_function + 203
     frame #48: 0x000000010a621afe plugin_host`PyEval_EvalFrameEx + 20335
     frame #49: 0x000000010a61c9d8 plugin_host`PyEval_EvalCodeEx + 1622
     frame #50: 0x000000010a5ad8f1 plugin_host`function_call + 365
     frame #51: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #52: 0x000000010a59f5c9 plugin_host`method_call + 136
     frame #53: 0x000000010a58e1be plugin_host`PyObject_Call + 103
     frame #54: 0x000000010a624cbf plugin_host`PyEval_CallObjectWithKeywords + 93
     frame #55: 0x000000010a651ce9 plugin_host`t_bootstrap + 70
     frame #56: 0x00007fffbebf7aab libsystem_pthread.dylib`_pthread_body + 180
     frame #57: 0x00007fffbebf79f7 libsystem_pthread.dylib`_pthread_start + 286
     frame #58: 0x00007fffbebf7221 libsystem_pthread.dylib`thread_start + 13