Adding balanced parentheses to statements

Dear all

I am trying to use the Rewriter class to do something conceptually pretty straightforward to a source file. What I actually want to do is pretty specific, but if I could work out how to do the following it would be obvious how to solve my problem:

- for every Stmt node in the AST, replace the source code associated with the Stmt with:

     open(x) original_source_code_for_Stmt close(x)

where x is a number chosen uniquely for the Stmt node.

For instance, if the Stmt is the expression:

x + y

then I would like to end up with something like:

open(2) open(0) x close(0) + open(1) y close(1) close(2)

I hope my objective makes sense.

What I've tried to do is write a class that extends RecursiveASTVisitor, equipped with a Rewriter field, "TheRewriter", and an int field, "counter". The Rewriter should allow me to rewrite the source code. The counter is used to give each open and close a unique x. I'm assuming for now that all my code is in one source file.

I've tried two things:

   bool TraverseStmt(Stmt *s) {
     bool result = RecursiveASTVisitor::TraverseStmt(s);
     std::stringstream ss_start;
     ss_start << "open(" << counter << ")";
     TheRewriter.InsertTextAfter(s->getLocStart(), ss_start.str());
     std::stringstream ss_end;
     ss_end << "close(" << counter << ")";
     TheRewriter.InsertTextAfterToken(s->getLocEnd(), ss_end.str());
     counter++;
     return result;
   }

(This is what seemed natural to me from the clang APIs)

and also:

   bool TraverseStmt(Stmt *s) {
     bool result = RecursiveASTVisitor::TraverseStmt(s);
     std::stringstream ss_start;
     ss_start << "open(" << counter << ")";
     TheRewriter.InsertTextAfter(s->getLocStart(), ss_start.str());
     int offset = Lexer::MeasureTokenLength(s->getLocEnd(), TheRewriter.getSourceMgr(), TheRewriter.getLangOpts()) + 1;
     auto END1 = s->getLocEnd().getLocWithOffset(offset);
     std::stringstream ss_end;
     ss_end << "close(" << counter << ")";
     TheRewriter.InsertTextBefore(END1, ss_end.str());
     counter++;
     return result;
   }

(I came up with this after reading this source code from Robert Ankeney: Clang-tutorial/CIrewriter.cpp at master · loarabia/Clang-tutorial · GitHub)

However, neither work - my parentheses end up being unbalanced.

I think the problem may be that in an expression such as:

x + y

the sub-expressions "y" and "x + y" have the same end source location. But I cannot figure out how to get around this.

I'd be really grateful if anyone can give me a pointer here.

Best wishes

Ally Donaldson

Dear all

I am trying to use the Rewriter class to do something conceptually pretty
straightforward to a source file. What I actually want to do is pretty
specific, but if I could work out how to do the following it would be
obvious how to solve my problem:

- for every Stmt node in the AST, replace the source code associated with
the Stmt with:

    open(x) original_source_code_for_Stmt close(x)

where x is a number chosen uniquely for the Stmt node.

For instance, if the Stmt is the expression:

x + y

then I would like to end up with something like:

open(2) open(0) x close(0) + open(1) y close(1) close(2)

I hope my objective makes sense.

What I've tried to do is write a class that extends RecursiveASTVisitor,
equipped with a Rewriter field, "TheRewriter", and an int field,
"counter". The Rewriter should allow me to rewrite the source code. The
counter is used to give each open and close a unique x. I'm assuming for
now that all my code is in one source file.

I've tried two things:

  bool TraverseStmt(Stmt *s) {
    bool result = RecursiveASTVisitor::TraverseStmt(s);
    std::stringstream ss_start;
    ss_start << "open(" << counter << ")";
    TheRewriter.InsertTextAfter(s->getLocStart(), ss_start.str());
    std::stringstream ss_end;
    ss_end << "close(" << counter << ")";
    TheRewriter.InsertTextAfterToken(s->getLocEnd(), ss_end.str());

InsertTextAfter and InsertTextAfterToken return a bool indicating whether
the insertion was possible (false on success, true on failure). You could
use that to isolate which particular call is failing.

That said, the only cases where the above calls would currently fail are if
they're given a location for which TheRewriter.isRewritable returns false.
That only happens if the location provided is within a macro expansion.
Could that be the case in your failing examples?

Thanks for this suggestion. However, the calls do not fail: all the “open” and “close” strings are emitted. They are just not balanced. For example, I see this: open(8)(open(0)open(1)open(7)d close(1)close(0)+ open(6)(open(2)open(3)open(5)d close(3)close(2)/ open(4)2)close(5)close(4))close(7)close(6);close(8) for the expression: (d + (d / 2)) So you can see that all the opens have matching closes. It’s just that they are not balanced. Does this bring anything else to mind? Many thanks Ally

Dear Richard

Dear all

I am trying to use the Rewriter class to do something conceptually pretty
straightforward to a source file. What I actually want to do is pretty
specific, but if I could work out how to do the following it would be
obvious how to solve my problem:

- for every Stmt node in the AST, replace the source code associated with
the Stmt with:

    open(x) original_source_code_for_Stmt close(x)

where x is a number chosen uniquely for the Stmt node.

For instance, if the Stmt is the expression:

x + y

then I would like to end up with something like:

open(2) open(0) x close(0) + open(1) y close(1) close(2)

I hope my objective makes sense.

What I've tried to do is write a class that extends RecursiveASTVisitor,
equipped with a Rewriter field, "TheRewriter", and an int field,
"counter". The Rewriter should allow me to rewrite the source code. The
counter is used to give each open and close a unique x. I'm assuming for
now that all my code is in one source file.

I've tried two things:

  bool TraverseStmt(Stmt *s) {
    bool result = RecursiveASTVisitor::TraverseStmt(s);
    std::stringstream ss_start;
    ss_start << "open(" << counter << ")";
    TheRewriter.InsertTextAfter(s->getLocStart(), ss_start.str());
    std::stringstream ss_end;
    ss_end << "close(" << counter << ")";
    TheRewriter.InsertTextAfterToken(s->getLocEnd(), ss_end.str());

InsertTextAfter and InsertTextAfterToken return a bool indicating
whether the insertion was possible (false on success, true on failure). You
could use that to isolate which particular call is failing.

Thanks for this suggestion. However, the calls do not fail: all the
"open" and "close" strings are emitted. They are just not balanced. For
example, I see this:

    open(8)(open(0)open(1)open(7)d close(1)close(0)+
open(6)(open(2)open(3)open(5)d close(3)close(2)/
open(4)2)close(5)close(4))close(7)close(6);close(8)

for the expression:

   (d + (d / 2))

So you can see that all the opens have matching closes. It's just that
they are not balanced.

Does this bring anything else to mind?

Oh, I see what you mean now. The problem seems to be that you're adding the
"open" string for the outer statement after the open string for its first
substatement. Try this:

InsertTextAfter(s->getLocStart(), ...);
TraverseStmt(s);
InsertTextAfterToken(s->getLocEnd(), ...);

(or change your first InsertTextAfter to InsertTextBefore).

I opted for your first suggestion, and it worked like a charm - many thanks! Best wishes Ally