Integrating clang-based refactoring tools into IDEs - best practices?

Hello all,

for KDevelop, we have a GSOC student this summer doing a R&D project on how we
could leverage libTooling and friends to create refactoring tools for C++
code. I'm pretty sure that this is the correct way in the long run, but I have
some trouble with the lack of API and ABI stability in the C++ libraries of
LLVM, most notably libTooling here.

I was thinking about a separate process that works only against a very
specific LLVM version. Thus, if distributors update LLVM, and the stuff does
not compile anymore, we only loose the refactoring but not the whole language
support (based on clang-c).
With this approach though, the cached parse data cannot be shared between
language support and refactoring...

How do other IDEs handle this situation? Is there maybe an external
libTooling-c shim or similar that we could leverage?

Bye

Hello all,

for KDevelop, we have a GSOC student this summer doing a R&D project on how we
could leverage libTooling and friends to create refactoring tools for C++
code. I’m pretty sure that this is the correct way in the long run, but I have
some trouble with the lack of API and ABI stability in the C++ libraries of
LLVM, most notably libTooling here.

I was thinking about a separate process that works only against a very
specific LLVM version. Thus, if distributors update LLVM, and the stuff does
not compile anymore, we only loose the refactoring but not the whole language
support (based on clang-c).
With this approach though, the cached parse data cannot be shared between
language support and refactoring…

How do other IDEs handle this situation? Is there maybe an external
libTooling-c shim or similar that we could leverage?

I think you’re the first IDE wanting to do that (so I’m super super excited and want to help you where I can :D)

The original plan was to basically get refactorings into libclang once they’re working well enough. To my knowledge we don’t have such refactorings yet - is the SOC project about building refactorings, too? :slight_smile:

For tools like clang-format or clang-tidy we have integrations that use the command line interfaces.

On a side note:
For API stability, I think recent changes are pretty wild as things move towards more and more C++11 features. Usually we find it not that much work to keep up with the C++ APIs.
ABI stability is of course a different problem - if you want to use the platform libs instead of just building your own version of clang’s libs into your product, I do believe exporting stuff through libclang is pretty much the only sensible option.

Hello all,

for KDevelop, we have a GSOC student this summer doing a R&D project on how
we could leverage libTooling and friends to create refactoring tools for
C++ code. I'm pretty sure that this is the correct way in the long run, but
I have some trouble with the lack of API and ABI stability in the C++
libraries of LLVM, most notably libTooling here.

I just talked to a few people in #llvm and it appears that library ABI from
x.y.0 is supposed to be same for x.y.1 and onwards. So there seems to be a ABI
stability guarantee for patch releases. (Can someone from LLVM/Clang devs
confirm this? Tom Stellard told me it's indeed "official")

This combined with the fact that distros usually co-install LLVM/Clang minor
releases (3.5, 3.6, ...) should avoid us running in ABI issues, no?

I'd really appreciate if we could just use the libTooling API and share parsed
data (somehow...) and avoid the multi-process approach altogether. Maybe
that's indeed the better approach for our R&D project -- just use the API
directly, don't add a wrapper (yet).

PS: Sorry I didn't step in earlier on the KDevelop mailing list, but I'm
buried in "real" work at the moment

> Hello all,
>
> for KDevelop, we have a GSOC student this summer doing a R&D project on how
> we could leverage libTooling and friends to create refactoring tools for
> C++ code. I'm pretty sure that this is the correct way in the long run, but
> I have some trouble with the lack of API and ABI stability in the C++
> libraries of LLVM, most notably libTooling here.

I just talked to a few people in #llvm and it appears that library ABI from
x.y.0 is supposed to be same for x.y.1 and onwards. So there seems to be a ABI
stability guarantee for patch releases. (Can someone from LLVM/Clang devs
confirm this? Tom Stellard told me it's indeed "official")

See item #4: http://llvm.org/docs/HowToReleaseLLVM.html#release-patch-rules

This combined with the fact that distros usually co-install LLVM/Clang minor
releases (3.5, 3.6, ...) should avoid us running in ABI issues, no?

3.5 and 3.6 have different ABIs, so you will have to pick one to link against
unless your project has some kind of wrapper library that allows you to support
both versions at the same time.

-Tom

> > Hello all,
> >
> > for KDevelop, we have a GSOC student this summer doing a R&D project on
> > how
> > we could leverage libTooling and friends to create refactoring tools for
> > C++ code. I'm pretty sure that this is the correct way in the long run,
> > but
> > I have some trouble with the lack of API and ABI stability in the C++
> > libraries of LLVM, most notably libTooling here.
>
> I just talked to a few people in #llvm and it appears that library ABI
> from
> x.y.0 is supposed to be same for x.y.1 and onwards. So there seems to be a
> ABI stability guarantee for patch releases. (Can someone from LLVM/Clang
> devs confirm this? Tom Stellard told me it's indeed "official")

See item #4: http://llvm.org/docs/HowToReleaseLLVM.html#release-patch-rules

> This combined with the fact that distros usually co-install LLVM/Clang
> minor releases (3.5, 3.6, ...) should avoid us running in ABI issues, no?
3.5 and 3.6 have different ABIs, so you will have to pick one to link
against unless your project has some kind of wrapper library that allows
you to support both versions at the same time.

Right, that's what I was referring to. We pick either 3.5 or 3.6 as dependency
at compile time. Updates to the library then wouldn't cause issues for us,
since patch releases are supposed to be ABI compatible.

(Personally I didn't know there was such a guarantee in LLVM at all. Good to
know!)

In article <CAOsfVv=xwTSzkHsGomikvdG4EWJorCJVJx5nz695TR5vYaQ8Gw@mail.gmail.com>,
    Manuel Klimek <klimek@google.com> writes:

The original plan was to basically get refactorings into libclang once
they're working well enough. To my knowledge we don't have such
refactorings yet - is the SOC project about building refactorings, too? :slight_smile:

Every time I've asked about where to put refactoring tools, I'm always
told "clang-tidy", but this doesn't make the refactorings usable in
any context outside of clang-tidy.

Now I'm hearing that there was some plan to have them in a library all
along?

In article <CAOsfVv=xwTSzkHsGomikvdG4EWJorCJVJx5nz695TR5vYaQ8Gw@mail.gmail.com>,
Manuel Klimek <klimek@google.com> writes:

The original plan was to basically get refactorings into libclang once
they’re working well enough. To my knowledge we don’t have such
refactorings yet - is the SOC project about building refactorings, too? :slight_smile:

Every time I’ve asked about where to put refactoring tools, I’m always
told “clang-tidy”,

I’d fine that very curious… (when I say refactoring I usually mean “Fowler-style”)

In article <CAOsfVvnRwj36LrGnCRv4y7hi9S+ioEzyJNxaEc=aM-WmLNDrHA@mail.gmail.com>,
    Manuel Klimek <klimek@google.com> writes:

> Every time I've asked about where to put refactoring tools, I'm always
> told "clang-tidy",

I'd fine that very curious... (when I say refactoring I usually mean
"Fowler-style")

Ah, so you're thinking of very general refactorings like Extract
Method, Inline Method, etc. Over the years I've written up some
refactorings in Fowler style that are specific to C/C++:
<https://legalizeadulthood.wordpress.com/category/refactoring/>

The refactorings I've been doing in clang-tidy are maybe what you
would call "micro refactorings":

if (b == true) => if (b)

...and yet, these sorts of things are *very* useful from an IDE. Yes,
you can run clang-tidy and mark these as "fixups" that are available
by applying the suggested patch from clang-tidy, but both the micro
refactorings and the Fowler style ones seem like they'd be better off
in a library that was invoked in a shorter turnaround cycle from an
IDE. Do you agree?

clang-tidy is batch oriented. IDEs are interactive.

In article <CAOsfVvnRwj36LrGnCRv4y7hi9S+ioEzyJNxaEc=aM-WmLNDrHA@mail.gmail.com>,
Manuel Klimek <klimek@google.com> writes:

Every time I’ve asked about where to put refactoring tools, I’m always
told “clang-tidy”,

I’d fine that very curious… (when I say refactoring I usually mean
“Fowler-style”)

Ah, so you’re thinking of very general refactorings like Extract
Method, Inline Method, etc. Over the years I’ve written up some
refactorings in Fowler style that are specific to C/C++:
<https://legalizeadulthood.wordpress.com/category/refactoring/>

The refactorings I’ve been doing in clang-tidy are maybe what you
would call “micro refactorings”:

if (b == true) => if (b)

…and yet, these sorts of things are very useful from an IDE. Yes,
you can run clang-tidy and mark these as “fixups” that are available
by applying the suggested patch from clang-tidy, but both the micro
refactorings and the Fowler style ones seem like they’d be better off
in a library that was invoked in a shorter turnaround cycle from an
IDE. Do you agree?

clang-tidy is batch oriented. IDEs are interactive.

I think that’s not true in general. clang-tidy, the command line tool, in its current form is batch oriented. I don’t think there’s a generic limitation to integrate clang-tidy into IDEs (we integrate it into our code review tool for example).

I’m curious. How is the integration of clang-tidy with the review tool working?
Something that is run automatically before the reviewer even sees the code or something that is run on-demand, suggestion-by-suggestion?
Do the clang-tidy changes stay in the same CL or are separate CLs generated and submitted automatically?

I’m curious. How is the integration of clang-tidy with the review tool working?
Something that is run automatically before the reviewer even sees the code or something that is run on-demand, suggestion-by-suggestion?

Do the clang-tidy changes stay in the same CL or are separate CLs generated and submitted automatically?

More details than you will probably want, but hey, you asked :smiley:
http://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/43322.pdf