Hello,
I’m using libclang and its Python bindings to build a binding generator to a large C++ library using pybind11. All is going well but I am experiencing a strange issue and I’m hoping someone on this list has an idea. Unfortunately, I’m not able to reproduce the issue on a small scale so I’ll just describe it as best I can and if anyone has any thoughts please share.
I have a single include file (“all_includes.h”) that includes all the headers of the C++ library. This file is parsed by libclang and I use the results to write pybind11 code via the libclang Python bindings. I don’t get any translation errors or warnings. In some cases, certain types (usually parameter declarations) get resolved to “int” instead of the actual type like “std::list”.
If my “all_inlcudes.h” has only the header file where the issue occurs, the types are resolved correctly. It’s only when I include all my headers (or a large number of them) does this strange behavior start to appear. I can’t figure out a root reason or pattern for it happening, the only thing I’ve noticed is it seems to happen with things in a namespace liked “std::list<>” or “TypeA::value”.
One other thing, the C++ library makes heavy use of class templates. In libclang, these class declarations have no children (i.e., constructors, methods, etc.). In order to generate the binding code I have to retrieve the cursor to the class template, write its code then replace the template parameters with a string replace method. Is there a reason these class declarations from class templates don’t resolve the children of the class template with the template parameters already in place? It would make the binding generation process much easier and robust.
For either of these issues, my next step is to rewrite my binding generation tool in C++ and use the clang libraries like LibTooling. They seem to provide more functionality anyway, but the Python version is so close I’d thought it’d be worth asking before I go down that path.
Thanks,
Trevor
Hello,
I’m using libclang and its Python bindings to build a binding generator to
a large C++ library using pybind11. All is going well but I am experiencing
a strange issue and I’m hoping someone on this list has an idea.
Unfortunately, I’m not able to reproduce the issue on a small scale so I’ll
just describe it as best I can and if anyone has any thoughts please share.
I have a single include file (“all_includes.h”) that includes all the
headers of the C++ library. This file is parsed by libclang and I use the
results to write pybind11 code via the libclang Python bindings. I don’t
get any translation errors or warnings. In some cases, certain types
(usually parameter declarations) get resolved to “int” instead of the
actual type like “std::list<TypeA>”.
What kind of behavior do you get when you use a command-line clang or other
C++ compiler with a source file that includes "all_includes.h"?
I don't know much about libclang's operation but here's a wild guess: are
you getting implicit int parameters for undeclared functions? If so it
would seem that libclang might think this is C source and not C++. Going
out on a big limb: maybe this is also why you have trouble with types using
the scope resolution operator? Are you giving a header file to libclang as
if it were a translation unit for compilation? If that's the case then
does the behavior change if you call it "all_includes.hpp" or
"all_includes.H"? Have you tried explicitly indicating the language w/" -x
"? See
https://clang.llvm.org/docs/ClangCommandLineReference.html#introduction
towards the end.
If my “all_inlcudes.h” has only the header file where the issue occurs,
the types are resolved correctly. It’s only when I include all my headers
(or a large number of them) does this strange behavior start to appear. I
can’t figure out a root reason or pattern for it happening, the only thing
I’ve noticed is it seems to happen with things in a namespace liked
“std::list<>” or “TypeA::value”.
Is it possible that these are from preprocessor macros and they're getting
omitted when some critical definitions are absent/present?
One other thing, the C++ library makes heavy use of class templates. In
libclang, these class declarations have no children (i.e., constructors,
methods, etc.). In order to generate the binding code I have to retrieve
the cursor to the class template, write its code then replace the template
parameters with a string replace method. Is there a reason these class
declarations from class templates don’t resolve the children of the class
template with the template parameters already in place? It would make the
binding generation process much easier and robust.
For either of these issues, my next step is to rewrite my binding
generation tool in C++ and use the clang libraries like LibTooling. They
seem to provide more functionality anyway, but the Python version is so
close I’d thought it’d be worth asking before I go down that path.
I would try to bisect the differences between your libclang usage and some
known-good compiler configuration. Are the definitions the same? "clang++
-dM -E -x c++ /dev/null > cmdline_defs" -- Is the preprocessed output the
same? "clang++ -E all_includes.h > all_includes.ii"