Include Default C++ paths

Clang Frontend
1

Hi,
   I am trying to parse a C++ code but clang doesn't recognize the C++
types. Consider the following code

#include <iostream>
#include <string>

int main()
{
    string str;
}

clang gives an error at <iostream> and doesn't recognize the string type. To
enable C++, I did "langOptions.CPlusPlus=1" and the code recognizes custom
C++ classes. So how do I include default C++ paths?

2

I assume you're using clang as a library, rather than actually going
through the clang driver? The driver's where all (most) of the header
search logic is. You can extract the paths by simply asking clang what
command it used to invoke the frontend (I forget offhand, but there is
a verbose/print-commands option to clang) & just pass those in. But if
you want to make a tool using clang as a library & have the same
header/lib discovery that the clang command line program has, you'd
need to duplicate or reuse the logic in the driver.

- David

3

(readding cfe-dev)

What I am trying to do is to make a small program which parses the AST and
does some rewriting to output modified code. How will I use the Driver in
this case? All I want to do is to parse the AST.

I don't know off-hand what the best way to integrate the Driver's
lib/header discovery is, just that that's where the logic is.

The focus of Clang development with regard to "tools" like the one you
described is the Tooling infrastructure (overview of options & a link
to libTooling-specific information can be found here:
http://clang.llvm.org/docs/Tooling.html ). I'm not sure if that meets
the "standalone execution" scenario that you seem to be going for -
it's ideally meant to integrate with code already building using an
existing build system that has been modified to generate a database of
compilation commands. I /think/ it should also work standalone & still
do all the driver-based discovery, but I could be wrong.

4

(readding cfe-dev)

What I am trying to do is to make a small program which parses the AST and
does some rewriting to output modified code. How will I use the Driver in
this case? All I want to do is to parse the AST.

I don’t know off-hand what the best way to integrate the Driver’s
lib/header discovery is, just that that’s where the logic is.

The focus of Clang development with regard to “tools” like the one you
described is the Tooling infrastructure (overview of options & a link
to libTooling-specific information can be found here:
http://clang.llvm.org/docs/Tooling.html ). I’m not sure if that meets
the “standalone execution” scenario that you seem to be going for -
it’s ideally meant to integrate with code already building using an
existing build system that has been modified to generate a database of
compilation commands. I /think/ it should also work standalone & still
do all the driver-based discovery, but I could be wrong.

Hi,
I am trying to parse a C++ code but clang doesn’t recognize the C++
types. Consider the following code

#include
#include

int main()
{
string str;
}

Note that you should qualify string with its namespace: std::string str;. Won’t help much if the headers cannot be located obviously…

– Matthieu

5

(readding cfe-dev)

> What I am trying to do is to make a small program which parses the AST
and
> does some rewriting to output modified code. How will I use the Driver in
> this case? All I want to do is to parse the AST.

I don't know off-hand what the best way to integrate the Driver's
lib/header discovery is, just that that's where the logic is.

The focus of Clang development with regard to "tools" like the one you
described is the Tooling infrastructure (overview of options & a link
to libTooling-specific information can be found here:
http://clang.llvm.org/docs/Tooling.html ). I'm not sure if that meets
the "standalone execution" scenario that you seem to be going for -
it's ideally meant to integrate with code already building using an
existing build system that has been modified to generate a database of
compilation commands. I /think/ it should also work standalone & still
do all the driver-based discovery, but I could be wrong.

It does not fully do the driver-based discovery, but it'll usually work
with the same flags that you provide to a normal clang call - and you can
always specify options after "--" if you build the tool like in the
tutorials.

Cheers,
/Manuel

6

I’ve been looking at the source code of the Driver. What I have understood till now is that the driver is created for the platform we are working on. The driver can then build a compilation object from a list of arguments. Inside this compilation object, there exists a toolchain specific to the platform (e.g. Linux in my case) which has the job of inserting correct system and C++ include paths. Is that correct? Let us say that I have build such a compilation object. How can I create a CompilerInvocation or CompilerInstance out of this such that it includes the information about all default C and C++ paths? I see no way of using the driver to just create an AST and parse it so the only way is to create a Driver and somehow create a CompilerInstance out of it.

7

I have also tried using createInvocationFromCommandLine and createInvocationFromArgs. Both of these basically take args to create a compiler invocation . What arguments can be given so that clang automatically turns into clang++? I mean, isn’t there a simple argument that can enable the compilation of C++ rather than C just like the variable CCCisCXX in the Driver class does?

8

I’ve done the following. I’ve taken the source of createInvocationFromCommandLine(). Edited it a little and added the following important line:

TheDriver.CCCIsCXX = true;

The CompilerInvocation instance now has all the arguments for C++. The only problem remaining is that the C++ include paths are added AFTER the input file. Therefore, they are not processed. Why is this happening? How can I get these paths BEFORE the input file?

9

Another problem that you are likely to run into is that there are some
headers like stddef.h which are looked for in a hardcoded path
relative to the binary itself, so you must either hardcode the path on
your system in some way, or run your tool only in particular
directories.

The problem that you are having is probably the #1 most annoying thing
for me about clang. When I was getting started with clang I spent more
time than I wish to admit on exactly what you are trying to do now,
and basically my conclusion was that Clang does not support this use
case currently (i.e., using clang as a library to simply parse an
arbitrary C++ file and then do stuff with the AST or whatever).
libTooling was not around back then, so libTooling might be a way to
accomplish what you want. You are going to waste a lot of time if you
try to manually do this though---I certainly did.

Someday, we may be able to say ParseFile("foo.cpp") which just gives
you the ASTContext for the parsed file, but we currently do not, and
it is a *really*, **really** hard problem to be able to make something
like that "just work".

A plugin is a way to get around this though, but your tool may or may
not fit into the plugin model. If all you want though is to get the
ASTContext and do something with it, then a plugin is the way to go
currently.

-- Sean Silva

10

My use case currently requires me to be able to parse the AST. As I mentioned, I almost have the problem solved. I modified the “createInvocationFromCommandLine” function and added “TheDriver.CCCIsCXX=true”. The only problem is that the args in the CompilerInvocation object created are not in the correct order. All the C++ include arguments are put AFTER the source file and so they are not working. The “createInvocationFromArgs” is doing this. Is there any way this can be remedied?

On a completely different note, Can this be done using Libtooling? If yes, Can you please provide a small example. There is rarely any documentation on LibTooling and the Tooling API is much harder to understand than normal Clang API which is much more intuitive.

Regards,
Adil

11

My use case currently requires me to be able to parse the AST. As I
mentioned, I almost have the problem solved. I modified the
"createInvocationFromCommandLine" function and added
"TheDriver.CCCIsCXX=true". The only problem is that the args in the
CompilerInvocation object created are not in the correct order. All the C++
include arguments are put AFTER the source file and so they are not working.
The "createInvocationFromArgs" is doing this. Is there any way this can be
remedied?

If you have copied the function why not just write some code to
rearrange things to be like you want?

   On a completely different note, Can this be done using Libtooling? If
yes, Can you please provide a small example. There is rarely any
documentation on LibTooling and the Tooling API is much harder to understand
than normal Clang API which is much more intuitive.

There is an example at <http://clang.llvm.org/docs/LibTooling.html&gt;\.

-- Sean Silva

12

I can’t modify “createFromArgs” function of CompilerInvocation. I am not sure why it’s putting C++ paths at the end. Can you think of any reason?

Are there any other examples on Tooling which explore some other Actions etc. I am especially looking for something that enables me to rewrite the code. I am very grateful for your help.

Regards,
Mohammad Adil

13

Hi all,

The problem that you are having is probably the #1 most annoying thing
for me about clang. When I was getting started with clang I spent more
time than I wish to admit on exactly what you are trying to do now,
and basically my conclusion was that Clang does not support this use
case currently (i.e., using clang as a library to simply parse an
arbitrary C++ file and then do stuff with the AST or whatever).
libTooling was not around back then, so libTooling might be a way to
accomplish what you want. You are going to waste a lot of time if you
try to manually do this though---I certainly did.

Same for me. Spent a far too long time on that particular issue.

Someday, we may be able to say ParseFile("foo.cpp") which just gives
you the ASTContext for the parsed file, but we currently do not, and
it is a *really*, **really** hard problem to be able to make something
like that "just work".

I'm thinking about shipping the lib/clang/X.Y/include with my tool to be independent of a clang installation.

I managed to parse and load an AST for a given file using a code like the following:

   void Compiler::parseFile(std::string InputFile) {
     const char *clangInclude = get_the_aforementioned_include_path();
     ::setenv("COMPILER_PATH", clangInclude, 0);
     const char *args = { "-fno-spell-checking", "-I", clangInclude,
         InputFile.c_str() };
     int nargs = sizeof(args) / sizeof(args[0]);

     // Initialize diagnostic engine
     clang::DiagnosticConsumer* DiagClient = new clang::TextDiagnosticPrinter(
         m_DiagnosticStream, m_DiagOpts);
     llvm::IntrusiveRefCntPtr<clang::DiagnosticIDs> DiagID(
         new clang::DiagnosticIDs());
     clang::DiagnosticsEngine *DiagEngine = new clang::DiagnosticsEngine(DiagID,
         DiagClient);
     llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diag = &DiagEngine;

     // Now parse the file to a Translation Unit
     clang::ASTUnit *TU = clang::ASTUnit::LoadFromCommandLine(&args[0], &args[nargs],
         diag, m_ClangResourcePath);

     // do whatever you want with the TU, for instance...

     // Dump all top-level definitions
     for (clang::ASTUnit::top_level_iterator i = TU->top_level_begin(), e = TU->top_level_end(); i != e; ++i) {
       clang::Decl *D = (*i);
       if (TU->getSourceManager().isFromMainFile(D->getLocation())) {
         DumpASTVisitor DumpVisitor;
         DumpVisitor.TraverseDecl(D);
       }
     }
  }

If anyone has a better way to do this, I would be happy to read about it :slight_smile:

Mehdi Amini

14

I can't modify "createFromArgs" function of CompilerInvocation. I am not
sure why it's putting C++ paths at the end. Can you think of any reason?

What do you mean "putting C++ paths at the end"?

Are there any other examples on Tooling which explore some other Actions
etc. I am especially looking for something that enables me to rewrite the
code. I am very grateful for your help.

Look at tools/clang-check for a simple example. There is another
example at <https://github.com/chisophugis/clang_plugin_example/blob/master/FindDependencies.cpp&gt;\.

You might also want to check out the clang-tools-extra repo with `git
clone http://llvm.org/git/clang-tools-extra.git\`\.

-- Sean Silva

15

Hi,
   I have attached a folder containing the code and a readme file. Can you
please take a look at it? It shouldn't take much time to see the problem.
If this problem is solved, I believe that the clang internal API can very
easily incorporate C++ paths too. I am very grateful for your help.

Regards,
Adil