python libclang

Hi,

I’m trying to use python binding to libclang to generate include graphs. I took the example program (bindings/python/examples/cindex/cindex-includes.py) and add support to read a JSON compilation database, therefore I can have include graph for a whole project.

Now, I’m running against the Clang project. And it fails… I compiled Clang with itself. Without any problem… My tool read the compile_commands.json generated by cmake and pass all arguments (except the compiler and the module). So it supposes to run the same as it was by clang… And this is not happen with any modules, it compiles all LLVM code and the majority of Clang.

Any hints how can I fix it? Or make it reproducible for a bug report?

Regards,
Laszlo

The related JSON file entry looks like this:

{
“directory”: “/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization”,
“command”: “/home/laszlona/Programming/llvm-svn/usr/local/bin/clang++ -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D_GNU_SOURCE -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fPIC -fvisibility-inlines-hidden -Wno-nested-anon-types -Wcovered-switch-default -fno-common -Woverloaded-virtual -Wcast-qual -fno-strict-aliasing -pedantic -Wno-long-long -Wall -W -Wno-unused-parameter -Wwrite-strings -fno-rtti -O2 -g -I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization -I/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization -I/home/laszlona/Programming/llvm-svn.src/tools/clang/include -I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/include -I/home/laszlona/Programming/llvm-svn.src/build/include -I/home/laszlona/Programming/llvm-svn.src/include -Wall -W -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -pedantic -Wno-long-long -fno-exceptions -o CMakeFiles/clangSerialization.dir/ASTWriterStmt.cpp.o -c /home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp”,
“file”: “/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp”
},

The error report looks like this:

python: /home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Sema/SemaInit.cpp:3153: clang::OverloadingResult TryRefInitWithConversionFunction(clang::Sema&, const clang::InitializedEntity&, const clang::InitializationKind&, clang::Expr*, bool, clang::InitializationSequence&): Assertion `!S.CompareReferenceRelationship(Initializer->getLocStart(), T1, T2, DerivedToBase, ObjCConversion, ObjCLifetimeConversion) && “Must have incompatible references when binding via conversion”’ failed.
libclang: crash detected during parsing: {
‘source_filename’ : ‘/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp’
‘command_line_args’ : [’-D_GNU_SOURCE’, ‘-D_DEBUG’, ‘-D__STDC_CONSTANT_MACROS’, ‘-D__STDC_FORMAT_MACROS’, ‘-D__STDC_LIMIT_MACROS’, ‘-D_GNU_SOURCE’, ‘-DCLANG_ENABLE_ARCMT’, ‘-DCLANG_ENABLE_REWRITER’, ‘-DCLANG_ENABLE_STATIC_ANALYZER’, ‘-fPIC’, ‘-fvisibility-inlines-hidden’, ‘-Wno-nested-anon-types’, ‘-Wcovered-switch-default’, ‘-fno-common’, ‘-Woverloaded-virtual’, ‘-Wcast-qual’, ‘-fno-strict-aliasing’, ‘-pedantic’, ‘-Wno-long-long’, ‘-Wall’, ‘-W’, ‘-Wno-unused-parameter’, ‘-Wwrite-strings’, ‘-fno-rtti’, ‘-O2’, ‘-g’, ‘-I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization’, ‘-I/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization’, ‘-I/home/laszlona/Programming/llvm-svn.src/tools/clang/include’, ‘-I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/include’, ‘-I/home/laszlona/Programming/llvm-svn.src/build/include’, ‘-I/home/laszlona/Programming/llvm-svn.src/include’, ‘-Wall’, ‘-W’, ‘-Wno-unused-parameter’, ‘-Wwrite-strings’, ‘-Wmissing-field-initializers’, ‘-pedantic’, ‘-Wno-long-long’, ‘-fno-exceptions’, ‘-o’, ‘CMakeFiles/clangSerialization.dir/ASTWriterStmt.cpp.o’, ‘-c’],
‘unsaved_files’ : [],
‘options’ : 0,
}
Traceback (most recent call last):
File “./bin/include_graph_to_gexf.py”, line 107, in
pool.apply(compile_module_to_queue, args=(q, m))
File “/usr/lib64/python2.7/multiprocessing/pool.py”, line 220, in apply
return self.apply_async(func, args, kwds).get()
File “/usr/lib64/python2.7/multiprocessing/pool.py”, line 528, in get
raise self._value
AssertionError
Exception ctypes.ArgumentError: "argument 1: <type ‘exceptions.AttributeError’>: ‘TranslationUnit’ object has noProcess Process-2:
Traceback (most recent call last):
File “/usr/lib64/python2.7/multiprocessing/process.py”, line 258, in _bootstrap
self.run()
File “/usr/lib64/python2.7/multiprocessing/process.py”, line 114, in run
self._target(*self._args, **self._kwargs)
File “./bin/include_graph_to_gexf.py”, line 76, in build_graph_from_queue
for e in iter(queue.get, ‘STOP’):
File “”, line 2, in get
File “/usr/lib64/python2.7/multiprocessing/managers.py”, line 759, in _callmethod
kind, result = conn.recv()
EOFError

While multiprocessing.Pool offers an alluring API and promises of faster
performance, it has many gotchas and can blow up in many ways. I suggest
getting your tool to work right without multiprocessing.Pool. Only then
should you attempt to use multiprocessing.Pool.

Hi,

I'm trying to use python binding to libclang to generate include graphs. I took the example program (bindings/python/examples/cindex/cindex-includes.py) and add support to read a JSON compilation database, therefore I can have include graph for a whole project.

Now, I'm running against the Clang project. And it fails... I compiled Clang with itself. Without any problem... My tool read the `compile_commands.json` generated by cmake and pass all arguments (except the compiler and the module). So it supposes to run the same as it was by clang... And this is not happen with any modules, it compiles all LLVM code and the majority of Clang.

Any hints how can I fix it? Or make it reproducible for a bug report?

The "c-index-test" tool can be used to test libclang on a source file, e.g
c-index-test -test-load-source all <arguments>
will parse using the given compiler arguments and emit information about the translation unit.
If you can reproduce a crash using some compiler arguments, then you can try creating a preprocessed file and reducing that.

i’ve already made a bug report on it…

http://llvm.org/bugs/show_bug.cgi?id=15274