Help with libTooling

We have found AST limitations with libClang and understand few developers are using this approach for parsing C++. We have been advised to switch to using libTooling, which now makes sense.

We have found limited documentation and are working our why though what it take to set up a project which uses libTooling. The only tutorials we have found are from several years ago. If newer ones exist it would be great to know about them. Searching for “libTooling” yields too many hits on “libtool”, which is obviously not what we are looking for.

I believe we have figured out the appropriate include files. Our current obstacle is figuring out what clang libraries we need. This seems to be order dependent, but I could be incorrect. We have link errors and trying to look up undefined symbols in nearly impossible.

We look forward to someone who can point us in the right direction as it seems we are missing a few basics.

Thanks,

Barbara
Co-Founder of DoxyPress

I assume you have found:
http://clang.llvm.org/docs/LibTooling.html

(which is the first hit for me when searching for “libtooling clang”)

Manuel,

Yes, we have read all of the links on clang.llvm for LibTooling. Most simply mention how to use the existing tools. There is simply nothing we can find which shows which clang libraries are required for libTooling.

The page you listed shows how to run ClangCheck, not how to link an third party application with libTooling. We are developing a new tool and libTooling seems to be our best shot.

The page I linked shows how to write the code. The libraries you need follow from the headers / classes you use.

This:
http://clang.llvm.org/docs/LibASTMatchersTutorial.html

should also be still mostly up to date - if it isn’t, patches are welcome :smiley:

Cheers,
/Manuel

I solved the problem of which libraries by not solving it; on Windows, my compile command looks like

cl /I\llvm\build\include /I\llvm\build\tools\clang\include /I\llvm\include /I\llvm\tools\clang\include /MTd /Zi compile.cpp \llvm\build\Debug\lib*.lib setargv.obj

I.e. just link all the libraries. Is there an equivalent of that on Unix?

Okay so it’s working for me but not for you, and the two obvious possible candidates for reasons why are that you’re using MinGW or that your program does more than mine currently does, and therefore requires more of the functionality of libTooling. To distinguish between those, can you try compiling your program in the way I’ve been compiling mine, with Microsoft C++ driven by a Windows batch file, and see if that makes any difference?

Okay, so that rules out one explanation. I don’t have a MinGW build of LLVM to try right now; will have a go at getting one up and running over the weekend. I do have a Linux one to experiment with. Tentative conclusions so far:

llvm-config doesn’t work once you need clang as well as LLVM; we’re basically on our own as far as that goes - even the include file directories will need to be specified by hand, so we can resign themselves to doing the rest by hand as well.

The libTooling tutorial suggests building a custom version of cmake and then using a GUI program to configure the build on the current machine, which obviously isn’t a winning strategy for distributable software.

As for actually trying to build clang-using stuff Linux, haven’t yet got past the include stage. Here’s where I am currently at:

a@a-VirtualBox:~$ gcc -I/llvm/include -I~/build/include -I~/llvm/tools/clang/include -I~/build/tools/clang/include ~/ayane/compiler/*.cpp
/home/a/ayane/compiler/main.cpp:9:47: fatal error: clang/Tooling/CommonOptionsParser.h: No such file or directory
#include <clang/Tooling/CommonOptionsParser.h>
^
compilation terminated.
a@a-VirtualBox:~$ find . -name CommonOptionsParser.h
./llvm/tools/clang/include/clang/Tooling/CommonOptionsParser.h

so as far as I can see, I’m specifying the necessary directory but it’s still not working, but I’m not very familiar with Linux so I could just be overlooking the obvious - can you see what I’m missing?

How did you get LLVM to build with MinGW? I get an error message when I try cmake -G “MinGW Makefiles”

"cmake --help" will list all available generators. You might need to use

cmake -G "Unix Makefiles"

even on MinGW

Csaba

Hi Barbara,

We’re using libTooling in a project and, after a few tries, the minimum libs required that we found was:

-lclangTooling -lclangDriver -lclangFrontend -lclangParse -lclangSerialization -lclangSema -lclangAnalysis -lclangEdit -lclangLex -lclangAST -lclangBasic -lLLVMBitReader -lLLVMOption -lLLVMMC -lLLVMMCParser -lLLVMSupport

The bold part you can get from llvm-config --libs mcparser option bitreader (it will actually return them + -lLLVMCore, which increases our static binary by ~9MB). We also append llvm-config --system-libs, for llvm’s dependencies.

For clang libs, you’ll have to write them by yourself as there is no way to request them from the system/pkgconfig (please, someone correct me if I’m wrong).

If you use autotools in your project, I can provide the m4 scripts to find those libraries.

Thank you,

Okay, that looks good, thanks. A small point:

Extract the MSYS zip file. Add /c/MSys to your bash profile or path.

Add /c/MinGW to your bash profile or path.

did you mean /c/msys/bin and /c/mingw/bin?

I ran the process overnight, and got this error

C:\llvm-mingw>\ninja\ninja
[1495/2504] Linking CXX static library lib\libLLVMSystemZAsmPrinter.a
FAILED: cmd.exe /C “cd . && C:\CMake\bin\cmake.exe -E remove lib\libLLVMSystemZAsmPrinter.a && c:\mingw64\bin\ar.exe cq lib\libLLVMSystemZAsmPrinter.a lib/Target/SystemZ/InstPrinter/CMakeFiles/LLVMSy
stemZAsmPrinter.dir/SystemZInstPrinter.cpp.obj && c:\mingw64\bin\ranlib.exe lib\libLLVMSystemZAsmPrinter.a && cd .”
c:\mingw64\bin\ranlib.exe: unable to rename ‘lib\libLLVMSystemZAsmPrinter.a’; reason: Permission denied
[1495/2504] Linking CXX static library lib\libLLVMSystemZCodeGen.a
ninja: build stopped: subcommand failed.

it doesn’t seem to be a deterministic error - restarted ninja just now and it seems to have picked up where it left off. I wonder if that’s correlated with the surprising slowness and system impact of mingw (compared to either Microsoft C++ on Windows or GCC on Linux)? I remember it being said that fork is slow on Windows, maybe ninja or GCC make heavy use of fork somehow, or something like that?

Csaba,

On Windows the Unix generator will not work. You must use the "Ninja" generator.

Barbara

Him

Our many thanks to Chandler, Daniel, Mikhail, Russell, and a few others at CppNow. It was suggested we look at clangTidy which helped us figure out what libraries we needed. The second part was adding --start-group in the makefile, which came from an example Jeff Trull presented at CppNow.

We are now about to successfully link DoxyPress with the libTooling libraries in Autotools! Of course the same technique will work in our pending CMake build files.

We are able to utilize all of the lib clang code we developed for parsing C++ in the new code using libTooling. Actually the new code is shorter, more efficient, more readable, and requires less contortions.

At the current time we have two items we have not been able to figure out how to detect in a FunctionDecl.

1 explicit
2 noexcept

Any ideas would be very helpful.

Thanks,

Barbara
Co-Founder of DoxyPress

Our many thanks to Chandler, Daniel, Mikhail, Russell, and a few others at
CppNow. It was suggested we look at clangTidy which helped us figure out
what libraries we needed. The second part was adding --start-group in the
makefile, which came from an example Jeff Trull presented at CppNow.

We are now about to successfully link DoxyPress with the libTooling
libraries in Autotools! Of course the same technique will work in our
pending CMake build files.

We are able to utilize all of the lib clang code we developed for parsing
C++ in the new code using libTooling. Actually the new code is shorter,
more efficient, more readable, and requires less contortions.

At the current time we have two items we have not been able to figure out
how to detect in a FunctionDecl.

1 explicit

Maybe CXXConstructorDecl::isExplicit() works for you there.

2 noexcept

Take a look at FunctionProtoType::getNoexceptSpec/getNoexceptExpr etc.

-- James