Wrong order of AST traversal?

Hello @ all,

I want to annotate some function and generate some additional properties for the annotated functions in an xml document. My problem is, that the AST traversal seems to traverse the attributes of the function after the traversal of the function has been finished. I could not find any valuable resources on this topic by searching via google. So I am addressing my question here.

(Since I do not know if file attachments are allowed I provide the files as comments.)

My test files is: (attributeTest.cpp)

void func1()
__attribute((annotate(R"(Test)"))) {
int a = 1 + 1;
}

clang-check gives me the correct hierarchy: clang-check --ast-dump attributeTest.cpp

TranslationUnitDecl 0x559b4c7dc598 <>

-TypedefDecl 0x559b4c7dce70 <> implicit __int128_t ‘__int128’
-BuiltinType 0x559b4c7dcb30 '__int128' -TypedefDecl 0x559b4c7dced8 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128' -BuiltinType 0x559b4c7dcb50 ‘unsigned __int128’
-TypedefDecl 0x559b4c7dd208 <> implicit __NSConstantString ‘__NSConstantString_tag’
-RecordType 0x559b4c7dcfb0 '__NSConstantString_tag' -CXXRecord 0x559b4c7dcf28 ‘__NSConstantString_tag’
-TypedefDecl 0x559b4c7dd2a0 <> implicit __builtin_ms_va_list ‘char *’
-PointerType 0x559b4c7dd260 'char *' -BuiltinType 0x559b4c7dc630 ‘char’
-TypedefDecl 0x559b4c818958 <> implicit __builtin_va_list ‘__va_list_tag [1]’
-ConstantArrayType 0x559b4c818900 '__va_list_tag [1]' 1 -RecordType 0x559b4c7dd380 ‘__va_list_tag’
-CXXRecord 0x559b4c7dd2f0 '__va_list_tag' -FunctionDecl 0x559b4c818a98 </home/msagebaum/Kaiserslautern/Programms/adDSLParser/temp/attributeTest.cpp:1:1, line:4:1> line:1:6 func1 ‘void ()’
-CompoundStmt 0x559b4c818cb0 <line:2:36, line:4:1>
-DeclStmt 0x559b4c818c98 <line:3:3, col:16> -VarDecl 0x559b4c818bd8 <col:3, col:15> col:7 a ‘int’ cinit
`-BinaryOperator 0x559b4c818c78 <col:11, col:15> ‘int’ ‘+’

-IntegerLiteral 0x559b4c818c38 col:11 ‘int’ 1
-IntegerLiteral 0x559b4c818c58 <col:15> 'int' 1 -AnnotateAttr 0x559b4c818b30 <line:2:14, col:32> “Test”

If I now run the file through my AST Visitor: (parserTest.cpp)

#include

#include “clang/AST/ASTConsumer.h”
#include “clang/AST/QualTypeNames.h”
#include “clang/AST/RecursiveASTVisitor.h”
#include “clang/Frontend/CompilerInstance.h”
#include “clang/Frontend/FrontendAction.h”
#include “clang/Tooling/CommonOptionsParser.h”
#include “clang/Tooling/Tooling.h”

#include <llvm/Support/CommandLine.h>

using namespace llvm;
using namespace clang;
using namespace clang::tooling;

class XMLGenerator : public RecursiveASTVisitor {
public:

bool TraverseFunctionDecl(FunctionDecl *declaration) {

std::cout << “Function start” << std::endl;

RecursiveASTVisitor::TraverseFunctionDecl(declaration);

std::cout << “Function end” << std::endl;
return true;
}

bool TraverseAnnotateAttr(AnnotateAttr *a) {
std::cout << “Annotation start” << std::endl;

RecursiveASTVisitor::TraverseAnnotateAttr(a);

std::cout << “Annotation end” << std::endl;
return true;
}
};

class XMLGeneratorClassConsumer : public clang::ASTConsumer {
public:
explicit XMLGeneratorClassConsumer()
: Visitor() {}

virtual void HandleTranslationUnit(clang::ASTContext &Context) {
Visitor.TraverseDecl(Context.getTranslationUnitDecl());
}
private:
XMLGenerator Visitor;
};

class XMLGeneratorClassAction : public clang::ASTFrontendAction {
public:
virtual std::unique_ptrclang::ASTConsumer CreateASTConsumer(
clang::CompilerInstance &Compiler, llvm::StringRef InFile) {
return std::unique_ptrclang::ASTConsumer(
new XMLGeneratorClassConsumer());
}
};

static llvm::cl::OptionCategory MyToolCategory(“my-tool options”);

int main(int argc, const char **argv) {
CommonOptionsParser OptionsParser(argc, argv, MyToolCategory);
ClangTool Tool(OptionsParser.getCompilations(),
OptionsParser.getSourcePathList());

return Tool.run(newFrontendActionFactory().get());
}

Compilation: (compile.sh)

g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -c parserTest.cpp -o parserTest.o -I/usr/include
g++ -g -O0 -Wall -pedantic -std=c++11 -fno-rtti -o parserTest.exe parserTest.o -L/usr/lib64 -Wl,–start-group -lclangAnalysis -lclangParse -lclangEdit -lclangSema -lclangTooling -lclangDriver -lclangSerialization -lclangFrontend -lclangBasic -lclangASTMatchers -lclangAST -lclangLex -lLLVM-8 -Wl,–end-group -lpthread -lz

The result is:

Function start
Function end
Annotation start
Annotation end

What I would expect is:

Function start
Annotation start
Annotation end
Function end

My questions are now:
Is the order provided by my implementation the correct one or is the expected order the correct one?
Is there a way to change the visitor order such that attributes are visited inside of the function?
If not: Is there a way to get a list of attributes from ‘FunctionDecl’? (Could not find any methods in the api https://clang.llvm.org/doxygen/classclang_1_1FunctionDecl.html)

Thanks in advance for any help.

Cheers

Max

Hi Max,

You can get the attributes with Decl::attrs() method (also available for FunctionDecl).

Hello Ilya,

thank you for the fast reply. I do not know why I missed that.

Nevertheless, is the traversal order correct?

Cheers

Max

I believe it is intentional to have that traversal order. If we were to visit the attributes before the functiondecl itself, they would be meaningless without the context of where they are attached (Which *Attr doesn’t contain).

The argument that COULD be made is that it makes more sense to be traversed before the function body however.

I would agree the traversal order is wrong, attributes are logically attached to a function, i.e. they are “child nodes”.
Therefore, they should be traversed alongside others child nodes (function body, type, etc).

I looked quickly at RecursiveASTVisitor.h, and it looks like the
challenge is that attributes can be attached to other decls as well.

Attrs are traversed generically for all Decl after any dynamic-type traversal:
https://github.com/llvm/llvm-project/blob/master/clang/include/clang/AST/RecursiveASTVisitor.h#L707

So I think in order to change this, attribute traversal would need to
be custom-implemented for every kind of Decl.

- Kim

We already have a DEF_TRAVERSE_DECL macro that contains the boilerplate common to all Decl traversals, moving it there seems to give the desired behavior.
I’ve sent https://reviews.llvm.org/D64907 for review, let’s see whether Richard thinks it’s a good idea.

Nice, I hadn't seen that!

- Kim