clang 3.2 libclang: crash detected during reparsing

Hi,
Some example code for document editing I was using with clang 3.1 now segfaults on clang 3.2:

#include <clang-c/Index.h>
#include <string.h>

char DOCUMENT[] = “int main() { return 0; }\n”;

int main(int argc, char** argv) {
const char* args;
CXIndex ci = clang_createIndex(0,0);
CXTranslationUnit tu =
clang_parseTranslationUnit(ci, “main.cpp”, &args,
0, NULL, 0, clang_defaultEditingTranslationUnitOptions());
CXUnsavedFile file;
file.Filename = “main.cpp”;
file.Contents = DOCUMENT;
file.Length = strlen(DOCUMENT);
clang_reparseTranslationUnit(tu, 1, &file, clang_defaultEditingTranslationUnitOptions());
return 0;
}

Output is:
libclang: crash detected during reparsing

Segmentation fault (core dumped)

Am I doing something perverted or is this a bug?

Oliver

Fixed in r172632, thanks!

I just tried this again with the latest clang from svn. The original snippet works but the next lines in my original code are:

CXCursor cursor = clang_getTranslationUnitCursor(tu);
CXSourceRange sr = clang_getCursorExtent(cursor);
CXToken* tokens;
unsigned numtokens;
clang_tokenize(tu, sr, &tokens, &numtokens);
CXCursor* cursors = new CXCursor[numtokens];
clang_annotateTokens(tu, tokens, numtokens, cursors);

Appending even the first line of this (clang_getTranslationUnitCursor) to the example I gave earlier causes a segfault in the clang 3.3 library.

Oliver

I just tried this again with the latest clang from svn. The original snippet works but the next lines in my original code are:

CXCursor cursor = clang_getTranslationUnitCursor(tu);
CXSourceRange sr = clang_getCursorExtent(cursor);
CXToken* tokens;
unsigned numtokens;
clang_tokenize(tu, sr, &tokens, &numtokens);
CXCursor* cursors = new CXCursor[numtokens];
clang_annotateTokens(tu, tokens, numtokens, cursors);

Appending even the first line of this (clang_getTranslationUnitCursor) to the example I gave earlier causes a segfault in the clang 3.3 library.

Fixed in r178800. FYI, it looks like the CXTranslationUnit object that you get is null, so there was a problem during parsing.

I’ve also noticed this behaviour, where a null-TU gets returned when parsing failed. It’s usually unrecoverable for clang, and the only way to debug it is to turn on the option to write messages to stdout/stderr. I never got around to ask it, but is there a way to get these error messages even when the TU is null? I mean, the only way for e.g. an IDE to handle this is to silently ignore it, and that’s not very helpful…

– Erik.

I think maybe this message got lost somewhere, resending:

It seems that in the case of my example, the TU is null; indicating parsing has failed? What does it mean in this case for parsing to fail and why should it fail on such a case? Is there a new way to use this API? This example worked correctly in clang 3.1.

I don't know whether there is a policy to fail gracefully when a null TU is passed, but FWIW my null TU also causes a segfault with this basic block

for(unsigned i=0, n = clang_getNumDiagnostics(tu); i!=n; ++i) {
CXDiagnostic diag = clang_getDiagnostic(tu, i);
CXDiagnosticSeverity sev = clang_getDiagnosticSeverity(diag);
CXSourceLocation loc = clang_getDiagnosticLocation(diag);
CXString str = clang_formatDiagnostic(diag, clang_defaultDiagnosticDisplayOptions());
}

Regards,
Oliver

I just tried this again with the latest clang from svn. The original snippet works but the next lines in my original code are:

CXCursor cursor = clang_getTranslationUnitCursor(tu);
CXSourceRange sr = clang_getCursorExtent(cursor);
CXToken* tokens;
unsigned numtokens;
clang_tokenize(tu, sr, &tokens, &numtokens);
CXCursor* cursors = new CXCursor[numtokens];
clang_annotateTokens(tu, tokens, numtokens, cursors);

Appending even the first line of this (clang_getTranslationUnitCursor) to the example I gave earlier causes a segfault in the clang 3.3 library.

Fixed in r178800. FYI, it looks like the CXTranslationUnit object that you get is null, so there was a problem during parsing.

I’ve also noticed this behaviour, where a null-TU gets returned when parsing failed. It’s usually unrecoverable for clang, and the only way to debug it is to turn on the option to write messages to stdout/stderr. I never got around to ask it, but is there a way to get these error messages even when the TU is null? I mean, the only way for e.g. an IDE to handle this is to silently ignore it, and that’s not very helpful…

How about introducing a new CXTranslationUnit_ flag to pass to clang_parseTranslationUnit, which will cause that function to still return a CXTranslationUnit object on failure (maybe marked as erroneous or something) which the client will only use to get any stored diagnostic.
I think we also need some changes in ASTUnit because I believe that if clang::createInvocationFromCommandLine fails we don’t keep the diagnostic.

I think maybe this message got lost somewhere, resending:

It seems that in the case of my example, the TU is null; indicating parsing has failed? What does it mean in this case for parsing to fail and why should it fail on such a case? Is there a new way to use this API? This example worked correctly in clang 3.1.

Try calling clang_createIndex with second parameter set to 1 (for ‘displayDiagnostics’) and check the stderr output.

I don’t know whether there is a policy to fail gracefully when a null TU is passed, but FWIW my null TU also causes a segfault with this basic block

Thanks, fixed in r179121.

With displayDiagnostics, I get the following output on stderr:
error: error reading main.cpp

There is no main.cpp, the whole point is to have an in-memory document. I tried setting the string "main.cpp" in the call to clang_parseTranslationUnit to both an empty string and NULL, both of which cause the error to be suppressed but still return a null TU

Oliver

With displayDiagnostics, I get the following output on stderr:
error: error reading main.cpp

There is no main.cpp, the whole point is to have an in-memory document. I tried setting the string “main.cpp” in the call to clang_parseTranslationUnit to both an empty string and NULL, both of which cause the error to be suppressed but still return a null TU

In the source sample that you posted originally, you pass the CXUnsavedFile to clang_reparseTranslationUnit, not clang_parseTranslationUnit. I find it unlikely that this would work without a main.cpp file already present in the file system.