compile_commands.json for Windows/MSVC

Some of the clang based tools, such as clang-check, require a
compile_commands.json file which is generated by CMake. I tried
running Windows CMake to generate this file, but it issued an error
message and didn't write the file.

So I hacked one together using a Visual Studio project build log. It
gets me by, but lots of warnings are spewed about unrecognized command
line arguments.

Does anyone else have a better method for generating these? Maybe I
just need the right CMake incantation.

Make sure that you have the latest CMake and that you are generating projects as if clang was the compiler (i.e., not generating VS projects (although clang’s cl-emulation is getting quite good, but I don’t think it’s quite there yet)).

– Sean Silva

In article <CAHnXoan3PPNyDF=O1F43vuMjZLydMHJVkdPZ_jz7Fg3iJpXGJA@mail.gmail.com>,
    Sean Silva <silvas@purdue.edu> writes:

Make sure that you have the latest CMake

"c:\Program Files (x86)\CMake 2.8"\bin\cmake --version

cmake version 2.8.11.2

and that you are generating
projects as if clang was the compiler

I followed the directions here:
<http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html>

and ran this command:

cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../llvm

I get this message at the end:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_COMPILE_COMMANDS

...and sure enough, I don't have a compile_commands.json.

What did I do wrong?

What generator (-G) are you using?

-- Sean Silva

In article <CAHnXoak2COeWg6J7e+sHH005+RfE0GC=ZzwDTvXxXSOOK0Tghg@mail.gmail.com>,
    Sean Silva <silvas@purdue.edu> writes:

Searching the CMake source, I only see mentions of
CMAKE_EXPORT_COMPILE_COMMANDS in the make and ninja generators. The
feature is probably unimplemented for Visual Studio project files.

ninja works fairly well with the MSVC toolchain so you could try that.

Even if you get the cl.exe command lines, clang tools won't understand them
by default, and clang-cl is very new. We haven't used with clang tools,
and we haven't used it to build anything really big and complicated yet.

In article <CACs=ty+Bzw=1ZdNYhBRmSBDqt_y_O19dFd1Rb+hKj8EohZBykQ@mail.gmail.com>,
    Reid Kleckner <rnk@google.com> writes:

> > What generator (-G) are you using?
>
> When I originally created the build structure, I generated VS 2012
> project files.

Searching the CMake source, I only see mentions of
CMAKE_EXPORT_COMPILE_COMMANDS in the make and ninja generators. The
feature is probably unimplemented for Visual Studio project files.

OK, I'm a little confused. The page that explains how to generate
this json file doesn't mention anything about generators or -G.

Are the directions on the page simply assuming somethign that isn't true
in this case?

ninja works fairly well with the MSVC toolchain so you could try that.

I don't need to actually change the way I'm building the code -- I
still want VS projects for building the code. I just need to create
this JSON file.

As I understand it, this JSON file feeds clang all the necessary
include search paths and relevant preprocessor symbol definitions in
order to properly interpret the source code in a particular
compilation context.

AFAIK nobody implemented json compile command output for Windows yet (back in the days when I implemented it, clang was so unusable on windows that nobody cared). So, currently, your approach with scrapping the compile commands from the visual studio make files seems a way to go.

The next thing we’ll need is to be able to translate cl.exe command line options to clang’s. One way to do this is to use the ArgumentAdjuster to replace the cl.exe flags with the corresponding clang flags - again, somebody would need to implement this.

If nobody beats us to it, at some point we’ll most certainly get there, but currently, it’s not too high on our priority list. Thus, patches welcome :slight_smile:

Cheers,
/Manuel

AFAIK nobody implemented json compile command output for Windows yet (back
in the days when I implemented it, clang was so unusable on windows that
nobody cared). So, currently, your approach with scrapping the compile
commands from the visual studio make files seems a way to go.

As an alternative, you could add direct support to the tooling library for
querying Visual Studio for the commandline if there is a reasonable IPC
mechanism for doing that.

As another alternative, you could add a plugin to Visual Studio that caused
it to write out the json database whenever the project settings changed...
If there is a reasonable plugin layer to do that.

I would personally view both of these as strictly superior to teaching
CMake to do more things as they'll also work for non-CMake projects on
Windows. And if these aren't feasible, maybe some other solution that is
more generic.

I also think that whatever such system you could build, that integrates
cleanly on Windows, would be *very* welcome in the Clang repository, so
patches most definitely welcome there. (As in, I don't know how to do this,
but really really want it, so I'm crossing my fingers.)

The next thing we'll need is to be able to translate cl.exe command line
options to clang's. One way to do this is to use the ArgumentAdjuster to
replace the cl.exe flags with the corresponding clang flags - again,
somebody would need to implement this.

I don't think this is the right approach. We should just finish the Clang
driver's support that Reid and Hans have been working on. Honestly, it's
already pretty close.

I'd prefer to parse the VS project files and spit out a compilation
database. They are plain XML and as of VS2010 (I think) supposed to be
backward compatible, so it's a stable format.

I could probably help with VS IPC if that's something you want to
pursue, but it's usually pretty painful and less stable than just
interpreting the project file format.

I agree that CMake should be kept out of this loop; there are LOTS of
Windows-only projects maintaining their build systems as Visual Studio
solutions.

- Kim

My problem with this is that it adds an extra step every time the project
file changes. If you forget it, your tool begins to misbehave in
potentially surprising ways.

Maybe you could make the logic involve parsing and spitting out, but
embedding it into a plugin that gets notified when VS sees the project file
change? (I clearly have no idea how this stuff works...)

I see your concern. Yeah, it could probably be driven from a plug-in
of some kind, I don't know the details either. It thought it was messy
when I looked at it last a few years ago :slight_smile:

Isn't it just a dependency problem, though? If the modified time of
any vcxproj > the modified time of compile_commands.json it needs to
be regenerated? That way it would pick up changes that happen outside
of Visual Studio, e.g. a version control update, manual edit,
generated projects.

Actually, I started digging into Tooling yesterday, and maybe
compile_commands.json doesn't even need to be saved out. What if there
was code in Tooling to transform a Visual Studio solution into a
CompilationDatabase implementation? It would take longer to parse,
probably, but it should be a pretty direct translation.

- Kim

I think that'd be basically the same as some IPC solution (and probably
better, as msbuild seems pretty well defined / stable these days). Patches
for a CompilationDatabase implementation that just reads from msbuild would
be super-welcome :smiley:

For future reference, is there a basic XML parser available in the
LLVM/Clang tree?

- Kim

>
> I think that'd be basically the same as some IPC solution (and probably
> better, as msbuild seems pretty well defined / stable these days).
Patches
> for a CompilationDatabase implementation that just reads from msbuild
would
> be super-welcome :smiley:

For future reference, is there a basic XML parser available in the
LLVM/Clang tree?

No, but Clang already has an opt-in dependency on libxml....

They assume that you are using clang as the compiler for the project, which
currently isn't going to be true for VS projects.

-- Sean Silva

In article <CAGCO0KgVcAaZ2TMjVgrOamo+NAKHKrVuHNhacJ64E6i+QKO9nw@mail.gmail.com>,
    Chandler Carruth <chandlerc@google.com> writes:

As an alternative, you could add direct support to the tooling library for
querying Visual Studio for the commandline if there is a reasonable IPC
mechanism for doing that.

As another alternative, you could add a plugin to Visual Studio that caused
it to write out the json database whenever the project settings changed...
If there is a reasonable plugin layer to do that.

Both of these approaches are feasible with the first one probably
being easier.

I would personally view both of these as strictly superior to teaching
CMake to do more things as they'll also work for non-CMake projects on
Windows. And if these aren't feasible, maybe some other solution that is
more generic.

Yes, I would rather get a compile_commands.json for non-CMake projects
as well from Visual Studio.

I don't think this is the right approach. We should just finish the Clang
driver's support that Reid and Hans have been working on. Honestly, it's
already pretty close.

Yeah, with my hack I get usable results from libTooling as is, just a
bunch of warnings about arguments not understood.

Hello everyone.

I would like to know what is the status of this feature.
In fact I have been following clang on windows for some time.

i have installed the 3.4 version on Windows when it was released (thanks to
you all clang developer for this BTW)
( FYI I could not find the manual of clang modernize for 3.4 )
ASAIK the driver for visual is done.

And I have Cmake 2.8.12.1 (latest version) I haven't seen anything in the
changelogs for 2.8.12.2 neither 3.0 that is about this matter.

So I would like to know what I should do to get it working (I mean use the
compile_commands.json with visual generator of cmake).

Thank you for your help

+Hans

I think this depends on how much of VC’s command line arguments clang can already “eat”…

clang-cl should be able to "eat" all of them, but it will ignore a
bunch that are unsupported.

However, I don't think the clang tooling stuff is hooked up to
understand "cl-mode" arguments, since AFAIK, it doesn't invoke the
driver in the same way (but I could be wrong).

It might not be hard to make the tooling understand that the commands
from a compilation database should be interpreted in cl-mode, but I
haven't looked into it.

- Hans