AST Matchers tutorial

Clang Frontend
1

Hello, again!

As suggested by a few people, I have been writing up my experiences starting development on Clang, and specifically working with Clang-based refactoring tools. The repository where my loop converter will live doesn’t exist yet, so I will use the tutorial as a chance to start offering patches for review (they can be rebased to other locations later).

My intention was to begin assuming zero Clang experience and work towards building a slightly simplified version of the loop converter through a series of fully-explained patches to Clang to a unit-tested functional tool. Because the Getting Started page is aimed at people who want to use Clang rather than develop for it, I began with cloning clang+llvm from the git mirrors and preparing a cmake+ninja build.

What I have so far is currently available as a Google Doc here: https://docs.google.com/document/d/1oTkVLhCdRJUEH1_LDaQdXqe8-aOqT5GLDL9e4MhoFF8/edit

If enough people think that the tutorial is useful, I can eventually convert it to a nicer format (i.e. HTML with prettier code formatting). I expect to have made mistakes, especially in the section about obtaining and building Clang :slight_smile:

Thoughts?
-Sam

loop-converter-tutorial.patch (35 KB)

2

Thank you for writing this up! :slight_smile:

A couple things:

Since you're assuming zero Clang experience, I think you can leave out
the git-svn stuff (since AFAIK that is only relevant for people with
commit access). A simpler way to just checkout LLVM/Clang is

$ git clone http://llvm.org/git/llvm.git
$ cd source/tools
$ git clone http://llvm.org/git/clang.git

The git-svn checkout stuff that you are doing looks kind of fearsome.
It's easier for a newcomer to just use a pure git workflow.

Also, you may want to touch on the scenario that a person already has
CMake installed. E.g., that it should be a certain version, needs to
be compiled with certain flags, etc.

Ninja dependency resolver.

I would call this "Ninja build tool".

You should probably link to the LLVM commandline library docs
<http://llvm.org/docs/CommandLine.html&gt; in the relevant part of the
document.

the loop’s index variable is compared against N - 1

generally aren't half-open loops comparing against N, and not N-1? e.g.
for (int I = 0, N = 10; I != N; ++I)

I think that just a little bit more explanation about the use of
FoldingSetNodeID in areSameExpr() would be good. (just explain roughly
what it does, links to docs, etc)

As far as your patch. My gut seems inclined to have this inside the
examples/ subdirectory, since the code is really just example code to
accompany the tutorial.

That's all for now. Thank you so much for the extensive docs!

--Sean.

3

Thank you for writing this up! :slight_smile:

A couple things:

Since you're assuming zero Clang experience, I think you can leave out
the git-svn stuff (since AFAIK that is only relevant for people with
commit access). A simpler way to just checkout LLVM/Clang is

$ git clone http://llvm.org/git/llvm.git
$ cd source/tools
$ git clone http://llvm.org/git/clang.git

The git-svn checkout stuff that you are doing looks kind of fearsome.
It's easier for a newcomer to just use pure git workflow.

Also, you may want to touch on the scenario that a person already has
CMake installed. E.g., that it should be a certain version, needs to
be compiled with certain flags, etc.

Ninja dependency resolver.

I would call this "Ninja build tool".

You should probably link to the LLVM commandline library docs
<http://llvm.org/docs/CommandLine.html&gt; in the relevant part of the
document.

the loop’s index variable is compared against N - 1

generally aren't half-open loops comparing against N, and not N-1? e.g.
for (int I = 0, N = 10; I != N; ++I)

I think that just a little bit more explanation about the use of
FoldingSetNodeID in areSameExpr() would be good. (just explain roughly
what it does, links to docs, etc)

4

Nice! I *love* it. I have some stuff in the pipeline that will be
pretty orthogonal to what you wrote, but I also have to simply admit
defeat to your writing :slight_smile:

5

I’ve enabled comments on the Google Doc to help avoid polluting the mailing list too much :slight_smile:

6

On another discussion thread you mentioned that CMake needed to be compiled
with an extra flag. I don't happen to know the details beyond what is
written, since I set up my environment once, two months ago :slight_smile:
Would someone be able to list what's necessary and when?

I believe that you need CMake >=2.8.8 for Ninja, and currently need to
build CMake with -DCMAKE_ENABLE_NINJA on Mac (although I think the
bleeding edge version doesn't need this). I still haven't found a
time/place to actually follow all the steps in the tutorial (I've only
read it), but once I do (it might be a while unfortunately), I'll tell
you if anything didn't go well.

Actually, this really is intended as the start of the loop migration tool
I've been working on. I didn't just want to dump 3000 lines of code and
tests in one patch, though this set is far more fine-grained than I would
use for future patches. That aside, it would make sense for the tutorial to
live in examples/.

I think that it is perfectly fine to have the code for the tutorial be
in examples/, while the "real tool" would be somewhere else.

--Sean Silva