indexer plugin

Hello dear clang hackers,

I have recently begun using rtags to get some more grip on my code when working it in emacs. It helps me a lot, but it also uses a lot of resources while indexing my codebase. Recently I have been wondering if all that power going into parsing the code can be re-used to actually compile it. I was thinking that one could just tell the compiler to create some kind of index file along with the product as is possible for dependencies already.

Is there anything like that? I.e. planned for clang, or as a plugin somewhere? How hard would it be to make what rtags uses into such a plugin?

Best

Daniel

PS: please reply to me in person, as I am not currently subscribing this list.

+luke for somebody who knows that area

Hello dear clang hackers,

I have recently begun using rtags to get some more grip on my code when working it in emacs. It helps me a lot, but it also uses a lot of resources while indexing my codebase.

We’re working hard towards C++ modules, which will hopefully solve some of the resource issues here.

Indexing a C++ project will produce a lot of data. It’s difficult to tell when you’re doing redundant work; for example, if two source files include the same header, it’s not obvious whether the content of the header can be processed only once. As Manuel indicates, modules should help with this by limiting preprocessor shenanigans. Even then, you could still be in trouble if the header contains template definitions, since these are instantiated on demand (and your source files might make different demands). We’re looking at various ways to deal with this problem for Kythe, where we spend most of our C++ processing time not in Clang but instead in handling the raw output from our indexing tool. I think that you’d end up in the same situation (except maybe for very small codebases, or if your data model doesn’t store much detail)–you might get your index pieces for free, but the dominant cost of merging them all together will remain.

That’s interesting. I really have to check how rtags does this. It is fed with compilation commands for .o files (i.e. usually one per .cpp file). But I don’t know what it does for headers. I will get back once I have this figured out.

Thanks for answering.

Hello Daniel,

Maybe you’d like to check Mozilla’s DXR project [1] that indexes C and C++ code and provides a web interface to view and search code. It’s not as accurate as Google’s code search [2] for some templates, but DXR could be used as a reference because it’s open source.

[1]
https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/blob/master/dxr/plugins/clang/dxr-index.cpp

[2] https://code.google.com/p/chromium/codesearch

  • Jingyi