Interact with the preprocessor

dear cfe-list,

we need a way to verify the source code has some proprieties that
are lost after preprocessing, but impossible to detect directly from
source code without a full-fledged preprocessor.

For example, the `High-Integrity C++ Coding Standard Manual' has a rule
(14.10) that says that include directives can't contains path specifiers.

The code:

#define FILE "path/mylib.h"
#include FILE

does violate the rule, but I can't verify it automatically from the source
without a preprocessor because I need to know the meaning of FILE
(it might be defined in an other included file) and I can't verify it
afterwards because both the #define and the #include disappear.

Second example: it is always a bad idea (and it actually makes the code
ill formed) redefine a keyword. But preprocessors do not warn against it.

The code:

#define private public
class A { ... };

violates the rule, and we'd like to be able to intercept the action
of defining the macro in order to warn against this abuse of the
preprocessor. Again, the problem is undetectable in the preprocessed code.

Final example (again about registering the macro names): all inclusion
guards should be different inside the same project. Otherwise the inclusion
of one file will prevent to include another file. In this case we need
to insert our code in the preprocessor each time a #define is found in
order to see if the macro has already been defined elsewhere.

Is the clang preprocessor easy to interact/interface with? As far we could
understand, the answer should be negative because clang's preprocessor
seems to be tightly integrated with the subsequent parsing phases, but
we could be wrong (actually, we _hope_ to be wrong), so I asked here.

Thanks for any insight.
pb

Paolo Bolzoni wrote:

dear cfe-list,

we need a way to verify the source code has some proprieties that
are lost after preprocessing, but impossible to detect directly from
source code without a full-fledged preprocessor.

For example, the `High-Integrity C++ Coding Standard Manual' has a rule
(14.10) that says that include directives can't contains path specifiers.
  

Poor Boosters. :frowning:

Is the clang preprocessor easy to interact/interface with? As far we could
understand, the answer should be negative because clang's preprocessor
seems to be tightly integrated with the subsequent parsing phases, but
we could be wrong (actually, we _hope_ to be wrong), so I asked here.
  

The preprocessor and the main lexer are pretty tightly integrated, but
the parser is completely separate. You can run some of your validation
steps as a run separate from actual compilation and replace the parser.
I'm pretty sure you have enough control over the PP to do what you want.

Sebastian

I am missing what is the best way you are suggesting me.

What does that replace the parser mean? I saw the clang::Preprocessor but I
missed the way to interact with its actions (no virtual functions for example)

Or you mean I should make a completely new parser using clang (as lib) that
makes the checks?

thanks,
pb

Paolo Bolzoni wrote:
> dear cfe-list,
> Is the clang preprocessor easy to interact/interface with? As far we
> could understand, the answer should be negative because clang's
> preprocessor seems to be tightly integrated with the subsequent

parsing

> phases, but we could be wrong (actually, we _hope_ to be wrong), so I
> asked here.
The preprocessor and the main lexer are pretty tightly integrated, but
the parser is completely separate. You can run some of your validation
steps as a run separate from actual compilation and replace the parser.
I'm pretty sure you have enough control over the PP to do what you want.

I am missing what is the best way you are suggesting me.

What does that replace the parser mean? I saw the clang::Preprocessor but

I

missed the way to interact with its actions (no virtual functions for
example)

Or you mean I should make a completely new parser using clang (as lib)

that

makes the checks?

Yes, that's what I mean. The things you listed in your mail don't require
any parsing at all; they're only about macros. So you don't really need to
reimplement anything. Just write a program that uses the Clang preprocessor
as a library.

Sebastian

I think it would be reasonable to just make this a builtin warning in the normal clang preprocessor.

-Chris

So I should make a clang::preprocessor, set the predefines, the headers
directories and so go on. I know how to.

And I should check the rules using the member functions of the preprocessor.
I saw the clang::PPCallbacks class, using it I can make my own functions that
are called when:
- a source file is entered (via #include) or exited (because an eof is found
  in a included file.)
- when an #indent or #sccs directive is found.
- when a #pragma comment is found.

This is useful, but using PPCallbacks I can't detect when a macro is expanded
to what.

The clang::preprocessor class itself has many member functions.
A part of the usual get/set functions for the various parts,
there are various Enter* and Handle* functions that seem needed for
implementation more than for user.

The function void Lex(Token&) returns the next token after all the phases,
LexNonComment() is similar but skips comments.

There are few methods clean representation of Tokens.

About he previous rule example, detect if the user is redefining a
keyword.
The solution is use both Lex() and LexUnexpandedToken() and check if there is
a keyword in the latter and not in the former?
Is there not a easy way to interact with the operation just like the other
PPCallbacks functions?
Am I just missing the obvious?

Thanks for all the trouble,
pb