[libclang] [python][request] Is clang_tokenize and clang_getTokenSpelling being added to the python bindings anytime soon?


I recently was able to make my – very messed up - c version to output the values of the cursors I wanted on libclang and very much appreciate to know if there is any chance that the methods I listed on the title of this e-mail clang_tokenize and clang_getTokenSpelling would be available anytime soon.

More specifically what I am looking for is this. If I recurse the tree using python interface I get at the very most to the best of my knowledge:

piece of source sample: int loren = 2 + 2;

Cursor representing the binary operator (using cursor.kind, cursor.spelling, cursor.displayname, cursor.location, cursor.hash):

CursorKind.BINARY_OPERATOR None <SourceLocation file ‘simple.c’, line 11, column 14> 3289119033
type: TypeKind.INT
canonical type: TypeKind.INT

As you can see, I cant distinguish what is the binary operator (I get None). And this apply for other cursors as well.

However running the C code snipet from libclang which I found somewhere on stackoverflow with my code I was able to extract the token and obtain such information with some effort (I get the null instead of None but since I have access to the tokens associated with the Cursor I can reach it).

The output is: BinaryOperator (null)||| Start: Line: 11 Column: 14 Offset: 103 |||token = 2 token = + token = 2 token = ;

See code highlighted here: http://paste2.org/p/2062173

I already have this Python code written in a personal branch and just need to integrate it with the official bindings. This involves a little refactoring to make the API nicer to consume, etc.

I'll try to submit a patch for review by the end of next week.

While I'm at it, I may also add support for the comment APIs recently added to libclang since they are somewhat related.


Thank you very much Gregory.

I just found it by accident and saw now that what I am looking for is indeed located there (https://raw.github.com/indygreg/clang/python_features/bindings/python/clang/cindex.py) I am very much forwarding looking for this.

Best Regards,

Carlos Andrade

2012/6/28 Gregory Szorc <gregory.szorc@gmail.com>