I am now using clang API in order to alter some C sources using my custom ASTVisitor, and I am struggling to make my CompilerInstance object to know about the path of default system headers.
I have a protoype that uses the object CompilerInstance to initialize all the important stuffs (DiagnosticsEngine, Preprocessor ...) and calls "clang::ParseAST" at the end to use my own visitor.
This prototype works well for simple file that do not use any system headers (e.g. "stdlib.h"), but when I want to use these, I get errors like:
$ tsc.c:34:10: fatal error: 'stdlib.h' file not found
$ #include <stdlib.h>
If I set "Verbose = 1" to the HeaderSearchOptions of my CompilerInstance object, I can see that indeed, my search list is empty:
$ #include "..." search starts here:
$ End of search list.
I tried to add the system headers manually using "HeaderSearchOptions.Add" but (1) this is not portable and (2) anyway I can't make that work either.
Could you please explain me more in details how Clang does to find the default system headers ?
I attach the source file of my simple prototype.
None of these solutions relieves you from building your binary at the same place at Clang is installed, or duplicate the Clang headers along your tool installation so that they are in PATH_TO_BIN/../lib/...
Or did I miss something at some point?
I tried to add the system headers manually using
"HeaderSearchOptions.Add" but (1) this is not portable and (2)
anyway I can't make that work either.
Could you please explain me more in details how Clang does to find
the default system headers ?
I attach the source file of my simple prototype.
initialization it end up by looking the headers from the place the binary is installed.
None of these solutions relieves you from building your binary at the same
place at Clang is installed, or duplicate the Clang headers along your tool
installation so that they are in PATH_TO_BIN/../lib/...
Or did I miss something at some point?
What you can do is slurp up the builtin headers as char*'s in your sources,
compile them into your binary, and use virtual files at runtime with
LibTooling (we're actually doing that for our internal mapreduce framework).
The problem with this approach, and the main reason Chandler has opposed
doing that for quite some time, is that when you see errors that reference
the builtin headers you'll not be able to open those files from your editor
(Chandler will jump in if I misrepresented his views
Thank you for all your answers and sorry for the late reaction.
Let we avoid any misunderstanding. I can compile Clang and I can compile my tool without any issue.
The issue with systems headers occurs when I execute my tool, and when my tool parses a C file (or preprocesses it, with a "Preprocessor" object).
If I sum up your answers, there is no "magic" way that would make Clang API know about the path of my system headers. I have to "hard-code" them.
I find that a bit weird as when I compile Clang from the svn and use it as a frontend for LLVM to compile my files, it finds the headers without any intervention from myself.
Moreover, I've noticed that if I use libclang as defined in "Index.h" it also finds these. In this case however, I have to pass an argument to the function "clang_createTranslationUnitFromSourceFile" (at least "-Xclang", even without any further parameters is OK). However I've quitted with using it as seems no to feature any routines to parse the AST node by node, modify it and deserialize it.
I haven't tried the tooling library yet, does it feature such possibilities ?
Thank you for all your answers and sorry for the late reaction.
Let we avoid any misunderstanding. I can compile Clang and I can compile
my tool without any issue.
The issue with systems headers occurs when I execute my tool, and when my
tool parses a C file (or preprocesses it, with a "Preprocessor" object).
If I sum up your answers, there is no "magic" way that would make Clang
API know about the path of my system headers.
That is true.
I have to "hard-code" them.
That is not true While not "magic", the way to do this is to use the
Driver (which is exactly what clang-the-compiler uses to find the system
and builtin includes).
I find that a bit weird as when I compile Clang from the svn and use it as
a frontend for LLVM to compile my files, it finds the headers without any
intervention from myself.
Moreover, I've noticed that if I use libclang as defined in "Index.h" it
also finds these. In this case however, I have to pass an argument to the
function "clang_createTranslationUnitFromSourceFile" (at least "-Xclang",
even without any further parameters is OK). However I've quitted with using
it as seems no to feature any routines to parse the AST node by node,
modify it and deserialize it.
I haven't tried the tooling library yet, does it feature such
possibilities ?
Yes. It uses the driver and a compilation database so it gets the exact
same flags your normal compile gets, which, combined, will lead to the
right headers being found in the right places.
Is -internal-isystem the flag that controls this? That seems to be what lit is using for %clang_cc1.
Thank you. I looked up into Clang's sources and found many relevant references to this option, right into the Driver.
@ Manuel
That is not true While not "magic", the way to do this is to use the Driver (which is exactly what clang-the-compiler uses to find the system and builtin includes).
That is right. The implementations of the Driver (and ToolChain) seem to feature some nice features I should use.
I guess that means I will have forget about any CompilerInstance object. Do you have any code snippet that uses the Driver ?
And by the way, will I still be able to use ASTParser() ?
@ Jordan
> Is -internal-isystem the flag that controls this? That seems to be what
lit is using for %clang_cc1.
Thank you. I looked up into Clang's sources and found many relevant
references to this option, right into the Driver.
@ Manuel
> That is not true While not "magic", the way to do this is to use the
Driver (which is exactly what clang-the-compiler uses to find the system
and builtin includes).
That is right. The implementations of the Driver (and ToolChain) seem to
feature some nice features I should use.
I guess that means I will have forget about any CompilerInstance object.
Do you have any code snippet that uses the Driver ?
And by the way, will I still be able to use ASTParser() ?
I am now trying to use the libtooling. I get some errors about some missing JSON compilation database, but I'll figure out.
In the meantime I'll try to make my program work as soon as possible using the quick and dirty solution, and refine latter using the libTooling.
And I have a new question: where is the correct place to send arguments (like "-internal-isystem") to Clang's frontend ?
I've tried "CompilerInstance.makeDiagnostics()" but it looks like my arguments are not taken into account.
I attach my source (the main starts at line 387 and I create the diagnostics engine at line 437).
I've checked in the sources of LibTooling (as kindly pointed out by Manuel) and I could find any other location.... Maybe I messed up with the ordering of the function calls.
By the way, in order to add system headers, I've figured out how to use "HeaderSearchOptions.AddPath()" the correct way, but I get some errors in the file "xmmint.h" related to the width of vector. I think I should pass some extra argument arguments to the frontend...
FYI, the error I get is
In file included from tsc.c:43:
In file included from /home/trouve/boulot/2011_compiler/usr/lib/clang/3.2/include/xmmintrin.h:31:
/home/trouve/boulot/2011_compiler/usr/lib/clang/3.2/include/mmintrin.h:46:12: error: invalid conversion between vector type '__m64' and integer type 'int' of different size
return (__m64)__builtin_ia32_vec_init_v2si(__i, 0);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/trouve/boulot/2011_compiler/usr/lib/clang/3.2/include/mmintrin.h:70:12: error: invalid conversion between vector type '__m64' and integer type 'int' of different size
return (__m64)__builtin_ia32_packsswb((__v4hi)__m1, (__v4hi)__m2);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(see