How to link to "more docs" for clang-tidy checks

RFC:
Often, clang-tidy diagnostics leave you standing in the rain - someone claims what you just did is a bad idea, but there is neither a fixit provided, nor do you understand why there might be a problem.

Especially as we delve into the realm of more, um, controversial checks (cpp-core-guidelines), links to more documentation would be sometimes beneficial.

The question is if / how we’d best link to more docs from clang-tidy diagnostics.

Requirements are:

  • for a single clang release, the docs should be static (we don’t want the docs to not reflect the checks)
  • for ToT, the docs shouldn’t change unless we either verify that the updates are purely doc-fixes, or the code has changed to reflect the new docs
  • links need to be short, otherwise diags get too chatty

One possible solution would be to create a simple link-forwarding service on llvm.org (some JS inside a doc file), that will rewrite links into which we can encode whatever we want (revision, link-stamp, whathaveyou) to the relevant upstream docs (mostly github pages at a certain git hash).

Thoughts? Am I crazy?
Cheers,
/Manuel

Hi!

I do agree that, it would be awesome to have direct links to docs. An alternative way of doing this would be to distribute the static documentation with the release, and have a utility script that translates the checker name to a link to the local documentation and opens it up in the browser. The downside is that the diag message would not contain the link directly, so the users might miss the possibility of looking at the documentation for the checkers unless they read about the doc viewer script first.

I think having short links in the diag messages while we want to avoid collisions might be a big challenge.

Regards,

Gabor

Hi!

I do agree that, it would be awesome to have direct links to docs. An alternative way of doing this would be to distribute the static documentation with the release, and have a utility script that translates the checker name to a link to the local documentation and opens it up in the browser. The downside is that the diag message would not contain the link directly, so the users might miss the possibility of looking at the documentation for the checkers unless they read about the doc viewer script first.

I think having short links in the diag messages while we want to avoid collisions might be a big challenge.

Would you suggest to bundle external docs (cppcoreguidelines) with our release?

RFC:
Often, clang-tidy diagnostics leave you standing in the rain - someone
claims what you just did is a bad idea, but there is neither a fixit
provided, nor do you understand why there might be a problem.

Especially as we delve into the realm of more, um, controversial checks
(cpp-core-guidelines), links to more documentation would be sometimes
beneficial.

I think this is a great idea, if we can work it.

The question is if / how we'd best link to more docs from clang-tidy
diagnostics.

Requirements are:
- for a single clang release, the docs should be static (we don't want the
docs to not reflect the checks)

Agreed.

- for ToT, the docs shouldn't change unless we either verify that the
updates are purely doc-fixes, or the code has changed to reflect the new
docs

Agreed.

- links need to be short, otherwise diags get too chatty

Agreed.

I would suggest adding on the following requirements:

- The links have to be something we control redirects to (so we can
consistently deal with fixing broken links without issuing a new
release of clang-tidy).
- The links have to be able to handle rebranding of diagnostics. For
instance, cert-dcl59-cpp would have a link to CERT's documentation,
but that should not be reflected in google-build-namespaces, which is
what provides the implementation for cert-dcl59-cpp.
- The links should be tied to the specific diagnostic instances
instead of the checker as a whole (so that one checker providing two
diagnostics can have separate links for each diagnostic).

One possible solution would be to create a simple link-forwarding service on
llvm.org (some JS inside a doc file), that will rewrite links into which we
can encode whatever we want (revision, link-stamp, whathaveyou) to the
relevant upstream docs (mostly github pages at a certain git hash).

Thoughts? Am I crazy?

I think this is a great idea! One question: how do you envision
updating the ToT documentation to be the static documentation? Is that
a manual process we go through as part of the release checklist, or is
that an automated process we go through as part of the release
checklist?

~Aaron

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAOsfVv=sC9yW4wZbKkvP9LyqEGvMXRCLgBwuN2OunqkVsrCiDQ@mail.gmail.com>,
    Manuel Klimek via cfe-dev <cfe-dev@lists.llvm.org> writes:

The question is if / how we'd best link to more docs from clang-tidy
diagnostics.

I was discussing with Alexander Kornienko about how to include the
documentation in the clang-tidy executable itself so that I could ask
clang-tidy to describe a particular check for me. If the
documentation were in clang-tidy's executable/distribution then I
would I need to worry about link forwarding services and so-on? Is
this a simpler solution?

Requirements are:
- for a single clang release, the docs should be static (we don't want the
docs to not reflect the checks)

Including docs in the distribution would do this.

- for ToT, the docs shouldn't change unless we either verify that the
updates are purely doc-fixes, or the code has changed to reflect the new
docs

I don't know what "ToT" means.

- links need to be short, otherwise diags get too chatty

One possible solution would be to create a simple link-forwarding service
on llvm.org (some JS inside a doc file), that will rewrite links into which
we can encode whatever we want (revision, link-stamp, whathaveyou) to the
relevant upstream docs (mostly github pages at a certain git hash).

If you don't include docs in the distribution but instead spit out
links, then yes you need a link forwarding service in order to get
stable URLs.

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAOsfVv=sC9yW4wZbKkvP9LyqEGvMXRCLgBwuN2OunqkVsrCiDQ@mail.gmail.com>,
    Manuel Klimek via cfe-dev <cfe-dev@lists.llvm.org> writes:

The question is if / how we'd best link to more docs from clang-tidy
diagnostics.

I was discussing with Alexander Kornienko about how to include the
documentation in the clang-tidy executable itself so that I could ask
clang-tidy to describe a particular check for me. If the
documentation were in clang-tidy's executable/distribution then I
would I need to worry about link forwarding services and so-on? Is
this a simpler solution?

Requirements are:
- for a single clang release, the docs should be static (we don't want the
docs to not reflect the checks)

Including docs in the distribution would do this.

Depending on the license for the documentation itself, we would have
to obtain permission from the copyright owners to do this, which could
severely limit what docs we can include.

- for ToT, the docs shouldn't change unless we either verify that the
updates are purely doc-fixes, or the code has changed to reflect the new
docs

I don't know what "ToT" means.

Tip of Tree (latest trunk svn version).

~Aaron

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAAt6xTuJ7pvnZk2LhjAZad=4340jA=rh43rSVNyRbMK5SK2QYw@mail.gmail.com>,
    Aaron Ballman via cfe-dev <cfe-dev@lists.llvm.org> writes:

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAAt6xTuJ7pvnZk2LhjAZad=4340jA=rh43rSVNyRbMK5SK2QYw@mail.gmail.com>,
    Aaron Ballman via cfe-dev <cfe-dev@lists.llvm.org> writes:

> In article
<CAOsfVv=sC9yW4wZbKkvP9LyqEGvMXRCLgBwuN2OunqkVsrCiDQ@mail.gmail.com>,
> Manuel Klimek via cfe-dev <cfe-dev@lists.llvm.org> writes:
>
>> Requirements are:
>> - for a single clang release, the docs should be static (we don't want
the
>> docs to not reflect the checks)
>
> Including docs in the distribution would do this.

Depending on the license for the documentation itself, we would have
to obtain permission from the copyright owners to do this, which could
severely limit what docs we can include.

I'm confused. We're talking about the documentation for clang-tidy
checks, which is wholly within the LLVM.org's ownership as it is part
of their source tree.

Why would we be limited in any respect on these docs?

I was under the impression that we were talking about *external*
documentation, such as C++ Core Guidelines or the CERT C++ Coding
Standard.

~Aaron

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAAt6xTvNGQNBYNj8i0T-jgk3urwuUtjFt6gbbT-tDRdnFm2Uiw@mail.gmail.com>,
    Aaron Ballman via cfe-dev <cfe-dev@lists.llvm.org> writes:

> I'm confused. We're talking about the documentation for clang-tidy
> checks, which is wholly within the LLVM.org's ownership as it is part
> of their source tree.
>
> Why would we be limited in any respect on these docs?

I was under the impression that we were talking about *external*
documentation, such as C++ Core Guidelines or the CERT C++ Coding
Standard.

Ah.

These would be supporting supplementary material to the clang-tidy
documentation.

For that, we're back to providing links and using a link redirection
mechanism I suppose.

In my email to Alexander, I was thinking along the lines of how cmake
provides command-line switches to print out all of the documentation
for everything that is in cmake. They don't need to link to external
3rd party documents, however.

I have discussed this somewhat with Alex in the past and I think we should wait for him to comment on this before doing anything. He is on vacation, but should be able to comment in about two weeks.

I think we should first solve the internal documentation problem, i.e. consider how we link to 3rd party documents later. For me, the important question is where the documentation should actually live. I would prefer it to live in the source files itself. This makes it easier to have a proper versioning of the documentation and ensure that code changes are accompanied by corresponding documentation changes. Furthermore, if we put the documentation into a string in the source file, we can actually make something like “clang-tidy -describe X” as well as “clang-tidy -verbose-messages” work (the former just displaying the documentation for check X, the latter printing out the full documentation of checks warning on specific errors).

I agree that it would additionally be useful to extract the documentation from the checks and put them behind a short link that we can attach to all the messages. I can see various ways to do this, but I think we should first decide whether it makes sense to put the documentation into (a string in) the source code itself.

I have discussed this somewhat with Alex in the past and I think we should
wait for him to comment on this before doing anything. He is on vacation,
but should be able to comment in about two weeks.

Strongly agreed.

I think we should first solve the internal documentation problem, i.e.
consider how we link to 3rd party documents later. For me, the important
question is where the documentation should actually live. I would prefer it
to live in the source files itself. This makes it easier to have a proper
versioning of the documentation and ensure that code changes are accompanied
by corresponding documentation changes. Furthermore, if we put the
documentation into a string in the source file, we can actually make
something like "clang-tidy -describe X" as well as "clang-tidy
-verbose-messages" work (the former just displaying the documentation for
check X, the latter printing out the full documentation of checks warning on
specific errors).

I agree that it would additionally be useful to extract the documentation
from the checks and put them behind a short link that we can attach to all
the messages. I can see various ways to do this, but I think we should first
decide whether it makes sense to put the documentation into (a string in)
the source code itself.

We solved this for Clang's attributes by table-generating the
documentation and generating the web view server-side. We've also
discussed using the same machinery to drive documentation for clang
driver options. I think we could use a similar thing for clang-tidy (I
am still in favor of table-generating a lot of the machinery that we
currently use python scripts to generate for us).

~Aaron

I have discussed this somewhat with Alex in the past and I think we should wait for him to comment on this before doing anything. He is on vacation, but should be able to comment in about two weeks.

I think we should first solve the internal documentation problem, i.e. consider how we link to 3rd party documents later. For me, the important question is where the documentation should actually live. I would prefer it to live in the source files itself. This makes it easier to have a proper versioning of the documentation and ensure that code changes are accompanied by corresponding documentation changes. Furthermore, if we put the documentation into a string in the source file, we can actually make something like “clang-tidy -describe X” as well as “clang-tidy -verbose-messages” work (the former just displaying the documentation for check X, the latter printing out the full documentation of checks warning on specific errors).

I agree that it would additionally be useful to extract the documentation from the checks and put them behind a short link that we can attach to all the messages. I can see various ways to do this, but I think we should first decide whether it makes sense to put the documentation into (a string in) the source code itself.

Note that this discussion is about links to external documentation, like the C++ core guidelines.

I have discussed this somewhat with Alex in the past and I think we
should wait for him to comment on this before doing anything. He is on
vacation, but should be able to comment in about two weeks.

I think we should first solve the internal documentation problem, i.e.
consider how we link to 3rd party documents later. For me, the important
question is where the documentation should actually live. I would prefer it
to live in the source files itself. This makes it easier to have a proper
versioning of the documentation and ensure that code changes are
accompanied by corresponding documentation changes. Furthermore, if we put
the documentation into a string in the source file, we can actually make
something like "clang-tidy -describe X" as well as "clang-tidy
-verbose-messages" work (the former just displaying the documentation for
check X, the latter printing out the full documentation of checks warning
on specific errors).

I agree that it would additionally be useful to extract the documentation
from the checks and put them behind a short link that we can attach to all
the messages. I can see various ways to do this, but I think we should
first decide whether it makes sense to put the documentation into (a string
in) the source code itself.

Note that this discussion is about links to external documentation, like
the C++ core guidelines.

Then I don't understand your requirements:
1. We cannot guarantee that external docs are static.
2. For ToT, we cannot control how external docs are changed.
3. Links don't need to be short as we can link to the external docs from
internal docs instead of linking to external docs directly.

We can if they are on github through the use of a git hash.

I think if we do (1), we can.

Whether we always want to do that is one question I’d like to discuss.

I have discussed this somewhat with Alex in the past and I think we
should wait for him to comment on this before doing anything. He is on
vacation, but should be able to comment in about two weeks.

I think we should first solve the internal documentation problem, i.e.
consider how we link to 3rd party documents later. For me, the important
question is where the documentation should actually live. I would prefer it
to live in the source files itself. This makes it easier to have a proper
versioning of the documentation and ensure that code changes are accompanied
by corresponding documentation changes. Furthermore, if we put the
documentation into a string in the source file, we can actually make
something like "clang-tidy -describe X" as well as "clang-tidy
-verbose-messages" work (the former just displaying the documentation for
check X, the latter printing out the full documentation of checks warning on
specific errors).

I agree that it would additionally be useful to extract the
documentation from the checks and put them behind a short link that we can
attach to all the messages. I can see various ways to do this, but I think
we should first decide whether it makes sense to put the documentation into
(a string in) the source code itself.

Note that this discussion is about links to external documentation, like
the C++ core guidelines.

Then I don't understand your requirements:
1. We cannot guarantee that external docs are static.

We can if they are on github through the use of a git hash.

For my needs at CERT, we use Confluence which has similar
functionality. Even a static versioning scheme (like what LLVM uses
with http://llvm.org/releases/3.7.0/docs/index.html) would be fine.

How would you envision the release process working? Who would be
responsible for getting the specific versions for each release? Would
we require an automated process for retrieving these versions before a
checker module can link to 3rd party docs?

2. For ToT, we cannot control how external docs are changed.

I think if we do (1), we can.

I think we would have to explicitly require that only versioned
documentation can be supported. It doesn't have to be hashes, it could
be any version-specific link. But it would be a problem to link to
unversioned documentation.

3. Links don't need to be short as we can link to the external docs from
internal docs instead of linking to external docs directly.

Whether we always want to do that is one question I'd like to discuss.

To know whether we want to do that or not depends on how you envision
surfacing these links. Are you thinking:

foo/bar/baz.cpp:3:24 warning: do not use const_cast
[cppcoreguidelines-pro-type=const-cast]
[https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md#-type3-dont-use-const_cast-to-cast-away-const-ie-at-all]

(Note, that link doesn't contain the hash, and so would be
considerably longer in practice for the static documentation.)

If so, I don't think we can get away with anything *but* shortened
links. Looking at that diagnostic, 21 characters for actual guidance
to the user, 21 characters of location information, 39 characters to
identify the checker, and 132 characters to tell the user where to go
for more documentation. That's a whole lot of text to throw out to a
user.

Or do you have other ideas on how to surface the feature?

~Aaron

[Please reply *only* to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAK_tg0xg7mYqndkS9W1p9g8+AEDphMRwM+ni_s69NyqcSvWGFw@mail.gmail.com>,
    Daniel Jasper via cfe-dev <cfe-dev@lists.llvm.org> writes:

I think we should first solve the internal documentation problem, i.e.
consider how we link to 3rd party documents later. For me, the important
question is where the documentation should actually live. I would prefer it
to live in the source files itself.

Me too.

When the recent commit was made that added the documentation for the
checks to the source tree, Alex and I discussed that it would be
better to have it extracted from source.

That way, you update the check and fix the comment in the source and
this ends up in the generated doxygen and the generated user
documentation all from a single source of truth.

[Please reply only to the list and do not include my email directly
in the To: or Cc: of your reply. Thanks.]

In article <CAK_tg0xg7mYqndkS9W1p9g8+AEDphMRwM+ni_s69NyqcSvWGFw@mail.gmail.com>,
Daniel Jasper via cfe-dev <cfe-dev@lists.llvm.org> writes:

I think we should first solve the internal documentation problem, i.e.
consider how we link to 3rd party documents later. For me, the important
question is where the documentation should actually live. I would prefer it
to live in the source files itself.

Me too.

Note that I started this thread specifically to talk about documentation that we don’t want to maintain as part of our own source code, because it is much too large for that (look at examples of the C++ core guidelines; every check has a large text with lots of examples why it is enforced).

I don’t think we want to maintain a copy of the C++ core guidelines in our source code.