AST Matcher for Decl subclass CilkSpawnDecl

Hi,

I am working on a modified version of clang for Intel Cilk Plus, from

based on clang 3.4.

They added

class CilkSpawnDecl : public Decl {
  // ...
  public:
  // ...
  /// \brief Returns the associated CapturedStmt.
  CapturedStmt *getCapturedStmt() { return CapturedSpawn; }
  const CapturedStmt *getCapturedStmt() const { return CapturedSpawn; }
  // ...
};

However there don't seem to be any releated AST matchers and I need to match

-CompoundStmt
`-CilkSpawnExpr
  >-CilkSpawnDecl
  > `-CapturedStmt
  > >-CapturedDecl
  > > `-ImplicitParamDecl
  > >-ExprWithCleanups
  > > `-CallExpr

------------^ this guy

which does not seem to be possible without a customized matcher for
CilkSpawnDecl.

I would like to define a AST Traversal Matcher, so I can do something
like this

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      hasCapturedStmt(
      ^ has(exprWithCleanups(has(callExpr))))))))
      `-- How?

I already defined cilkSpawnExpr and cilkSpawnDecl by using
VariadicDynCastAllOfMatcher which is working great.
Yet, I still need hasCapturedStmt and I can't get my head around the AST
matcher internals. I guess, I somehow need to define a matcher that
takes a CilkSpawnDecl node and returns true if getCapturedStmt() != 0, e.g.

match(CilkSpawnDecl const& spawnDecl) {
  if (spawnDecl.getCapturedStmt())
    return true;
  else
    return false;
}

But how do I define hasCapturedStmt() so that has(exprWithCleanups(...))
can act on the children of CapturedStmt?
I.e. how does an AST matcher hand down the list of children to the
following matcher expressions?

I looked at the macro AST_MATCHER_P but I'm unsure about ParamType,
ASTMatchFinder and BoundNodesTreeBuilder. Can I use that?

Thanks!

Georg

Hi,

I am working on a modified version of clang for Intel Cilk Plus, from
https://github.com/cilkplus
based on clang 3.4.

They added

class CilkSpawnDecl : public Decl {
// …
public:
// …
/// \brief Returns the associated CapturedStmt.
CapturedStmt *getCapturedStmt() { return CapturedSpawn; }
const CapturedStmt *getCapturedStmt() const { return CapturedSpawn; }
// …
};

However there don’t seem to be any releated AST matchers and I need to match

-CompoundStmt
`-CilkSpawnExpr

-CilkSpawnDecl
`-CapturedStmt

-CapturedDecl
-ImplicitParamDecl -ExprWithCleanups -CallExpr
------------^ this guy

which does not seem to be possible without a customized matcher for
CilkSpawnDecl.

I would like to define a AST Traversal Matcher, so I can do something
like this

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
hasCapturedStmt(
^ has(exprWithCleanups(has(callExpr))))))))
`-- How?

I already defined cilkSpawnExpr and cilkSpawnDecl by using
VariadicDynCastAllOfMatcher which is working great.
Yet, I still need hasCapturedStmt and I can’t get my head around the AST
matcher internals. I guess, I somehow need to define a matcher that
takes a CilkSpawnDecl node and returns true if getCapturedStmt() != 0, e.g.

match(CilkSpawnDecl const& spawnDecl) {
if (spawnDecl.getCapturedStmt())
return true;
else
return false;
}

But how do I define hasCapturedStmt() so that has(exprWithCleanups(…))
can act on the children of CapturedStmt?
I.e. how does an AST matcher hand down the list of children to the
following matcher expressions?

I looked at the macro AST_MATCHER_P but I’m unsure about ParamType,
ASTMatchFinder and BoundNodesTreeBuilder. Can I use that?

Yes. Take a look at for example:
http://reviews.llvm.org/diffusion/L/browse/cfe/trunk/include/clang/ASTMatchers/ASTMatchers.h;222237$3536

AST_MATCHER_P(ElaboratedType, hasQualifier,
internal::Matcher, InnerMatcher) {
if (const NestedNameSpecifier *Qualifier = Node.getQualifier())
return InnerMatcher.matches(*Qualifier, Finder, Builder);

return false;
}

You basically want to adapt this to your use case…

Note that a CilkSpawnDecl adaptor always contains a CapturedStmt AST. The captured stuff might not be a simple CallExpr because of "receivers" initialized by spawns. You may experiment the following cases:

_Cilk_spawn foo();
x = _Cilk_spawn bar();
T x = _Cilk_spawn foo(), y = bar(), z = _Cilk_spawn baz();

Thanks,
Wei

Thank you for the quick reply.

I found a partial solution to my problem. I dug down into the AST
matching code and noticed RecursiveASTVisitor did not recurse into
CapturedStmt in CilkSpawnDecl. So I did the patch

diff --git a/include/clang/AST/RecursiveASTVisitor.h
b/include/clang/AST/RecursiveASTVisitor.h
index 35eec69..a16e772 100755
--- a/include/clang/AST/RecursiveASTVisitor.h
+++ b/include/clang/AST/RecursiveASTVisitor.h
@@ -1305,6 +1305,7 @@ DEF_TRAVERSE_DECL(CapturedDecl, {
   })

DEF_TRAVERSE_DECL(CilkSpawnDecl, {
+ TRY_TO(TraverseStmt(D->getCapturedStmt()));
   })

DEF_TRAVERSE_DECL(EmptyDecl, { })

Again, this is what I want to match:
The AST:

-CompoundStmt
`-CilkSpawnExpr
  >-CilkSpawnDecl
  > `-CapturedStmt
  > >-CapturedDecl
  > > `-ImplicitParamDecl
  > >-ExprWithCleanups
  > > `-CallExpr

--------------^ this guy

After the patch, these match now:

// 1) matches
compoundStmt(
  has(cilkSpawnExpr(
   has(cilkSpawnDecl(
     has(capturedStmt()))))))

// 2) matches and binds
compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        hasDescendant(exprWithCleanups(
          has(callExpr().bind("callExpr"))))))))))

However this doesn't match, while it should IMHO.

// 3) does not match
compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
        has(capturedStmt(
            has(exprWithCleanups())))))))

Notice that I used hasDescendant(exprWithCleanups()) in 2) and
has(exprWithCleanups()) in 3).
ExprWithCleanups is a direct child of CapturedStmt. Any clue why 2)
doesn't match?

Thanks,
Georg

Thank you for the quick reply.

I found a partial solution to my problem. I dug down into the AST
matching code and noticed RecursiveASTVisitor did not recurse into
CapturedStmt in CilkSpawnDecl. So I did the patch

diff --git a/include/clang/AST/RecursiveASTVisitor.h
b/include/clang/AST/RecursiveASTVisitor.h
index 35eec69…a16e772 100755
— a/include/clang/AST/RecursiveASTVisitor.h
+++ b/include/clang/AST/RecursiveASTVisitor.h
@@ -1305,6 +1305,7 @@ DEF_TRAVERSE_DECL(CapturedDecl, {
})

DEF_TRAVERSE_DECL(CilkSpawnDecl, {

  • TRY_TO(TraverseStmt(D->getCapturedStmt()));
    })

DEF_TRAVERSE_DECL(EmptyDecl, { })

Again, this is what I want to match:
The AST:

-CompoundStmt
`-CilkSpawnExpr

-CilkSpawnDecl
`-CapturedStmt

-CapturedDecl
-ImplicitParamDecl -ExprWithCleanups -CallExpr
--------------^ this guy

After the patch, these match now:

// 1) matches
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt()))))))

// 2) matches and binds
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
hasDescendant(exprWithCleanups(
has(callExpr().bind(“callExpr”))))))))))

However this doesn’t match, while it should IMHO.

// 3) does not match
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(exprWithCleanups())))))))

Which one in this chain is the one that doesn’t match? (remove inner ones until it matches, send result).

Cheers,
/Manuel

These match:

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt()))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(decl()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        // for some reason has() does not work here
        hasDescendant(exprWithCleanups(
          has(callExpr().bind("callExpr")))))))))))

None of these match:

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(exprWithCleanups()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(expr()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(stmt()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(decl()))))))))

To me this looks like either
a) the ast-dump is incorrect and ExprWithCleanups is not a direct child
of CapturedStmt
or
b) there is a bug with CapturedStmt in the ast matching code /
RecursiveASTVisitor which makes ExprWithCleanups being found as
descendant but not as direct child.

Maybe a test-case against vanilla clang would be helpful? However I
don't know how to reproduce the AST without the cilk extensions.

Regards,
Georg

Thank you for the quick reply.

I found a partial solution to my problem. I dug down into the AST
matching code and noticed RecursiveASTVisitor did not recurse into
CapturedStmt in CilkSpawnDecl. So I did the patch

diff --git a/include/clang/AST/__RecursiveASTVisitor.h
b/include/clang/AST/__RecursiveASTVisitor.h
index 35eec69…a16e772 100755
— a/include/clang/AST/__RecursiveASTVisitor.h
+++ b/include/clang/AST/__RecursiveASTVisitor.h
@@ -1305,6 +1305,7 @@ DEF_TRAVERSE_DECL(__CapturedDecl, {
})

DEF_TRAVERSE_DECL(__CilkSpawnDecl, {

  • TRY_TO(TraverseStmt(D->__getCapturedStmt()));
    })

DEF_TRAVERSE_DECL(EmptyDecl, { })

Again, this is what I want to match:
The AST:

-CompoundStmt
`-CilkSpawnExpr

-CilkSpawnDecl
`-CapturedStmt

-CapturedDecl
-ImplicitParamDecl -ExprWithCleanups -CallExpr
--------------^ this guy

After the patch, these match now:

// 1) matches
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt()))))))

// 2) matches and binds
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
hasDescendant(exprWithCleanups(
has(callExpr().bind(“callExpr”
))))))))))

However this doesn’t match, while it should IMHO.

// 3) does not match
compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(exprWithCleanups())))))))

Which one in this chain is the one that doesn’t match? (remove inner
ones until it matches, send result).

These match:

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt()))))))

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(decl()))))))))

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
// for some reason has() does not work here
hasDescendant(exprWithCleanups(
has(callExpr().bind(“callExpr”)))))))))))

None of these match:

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(exprWithCleanups()))))))))

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(expr()))))))))

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(stmt()))))))))

compoundStmt(
has(cilkSpawnExpr(
has(cilkSpawnDecl(
has(capturedStmt(
has(decl()))))))))

To me this looks like either
a) the ast-dump is incorrect and ExprWithCleanups is not a direct child
of CapturedStmt
or
b) there is a bug with CapturedStmt in the ast matching code /
RecursiveASTVisitor which makes ExprWithCleanups being found as
descendant but not as direct child.

Maybe a test-case against vanilla clang would be helpful? However I
don’t know how to reproduce the AST without the cilk extensions.

The problem is that the RecursiveASTVisitor and the -ast-dump logic use completely different code paths… I still think this is probably a bug in one of the two.
+richardsmith, who knows more about this…

Just to prevent any duplicate work: remember cilkplus is based on clang
3.4. This might already be fixed in current clang.
I might dig into this deeper later. For now the hasDescendant match is
good enough for me at this point.

Regards,
Georg

>
> Thank you for the quick reply.
>
> I found a partial solution to my problem. I dug down into the AST
> matching code and noticed RecursiveASTVisitor did not recurse into
> CapturedStmt in CilkSpawnDecl. So I did the patch
>
>
> diff --git a/include/clang/AST/__RecursiveASTVisitor.h
> b/include/clang/AST/__RecursiveASTVisitor.h
> index 35eec69..a16e772 100755
> --- a/include/clang/AST/__RecursiveASTVisitor.h
> +++ b/include/clang/AST/__RecursiveASTVisitor.h
> @@ -1305,6 +1305,7 @@ DEF_TRAVERSE_DECL(__CapturedDecl, {
> })
>
> DEF_TRAVERSE_DECL(__CilkSpawnDecl, {
> + TRY_TO(TraverseStmt(D->__getCapturedStmt()));
> })
>
> DEF_TRAVERSE_DECL(EmptyDecl, { })
>
>
> Again, this is what I want to match:
> The AST:
> >-CompoundStmt
> > `-CilkSpawnExpr
> > >-CilkSpawnDecl
> > > `-CapturedStmt
> > > >-CapturedDecl
> > > > `-ImplicitParamDecl
> > > >-ExprWithCleanups
> > > > `-CallExpr
> --------------^ this guy
>
>
> After the patch, these match now:
>
> // 1) matches
> compoundStmt(
> has(cilkSpawnExpr(
> has(cilkSpawnDecl(
> has(capturedStmt()))))))
>
> // 2) matches and binds
> compoundStmt(
> has(cilkSpawnExpr(
> has(cilkSpawnDecl(
> has(capturedStmt(
> hasDescendant(__exprWithCleanups(
> has(callExpr().bind("callExpr"__))))))))))
>
>
> However this doesn't match, while it should IMHO.
>
> // 3) does not match
> compoundStmt(
> has(cilkSpawnExpr(
> has(cilkSpawnDecl(
> has(capturedStmt(
> has(exprWithCleanups())))))))
>
>
> Which one in this chain is the one that doesn't match? (remove inner
> ones until it matches, send result).

These match:

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt()))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(decl()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        // for some reason has() does not work here
        hasDescendant(exprWithCleanups(
          has(callExpr().bind("callExpr")))))))))))

None of these match:

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(exprWithCleanups()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(expr()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(stmt()))))))))

compoundStmt(
  has(cilkSpawnExpr(
    has(cilkSpawnDecl(
      has(capturedStmt(
        has(decl()))))))))

To me this looks like either
a) the ast-dump is incorrect and ExprWithCleanups is not a direct child
of CapturedStmt
or
b) there is a bug with CapturedStmt in the ast matching code /
RecursiveASTVisitor which makes ExprWithCleanups being found as
descendant but not as direct child.

Maybe a test-case against vanilla clang would be helpful? However I
don't know how to reproduce the AST without the cilk extensions.

The problem is that the RecursiveASTVisitor and the -ast-dump logic use
completely different code paths... I still think this is probably a bug in
one of the two.
+richardsmith, who knows more about this...

-ast-dump has no custom logic for CapturedStmt, so I'm surprised you're
getting the dump you provide above. I don't see that with clang trunk.
Perhaps this is something that the cilk folks changed.