RFC: Clang driver redesign (round 2)

Hi,

Thanks for all the replies. I've kind of summarised the main points of
contest so far below and tried to group supporters and opponents. If
I've grouped you incorrectly, please don't take it slanderously :slight_smile:

I've distilled the conversation so far into:
  * Accepted command line options should be easy to change, add and
    remove. Invoke time vs build time? Not decided.
  * Most options should be able to be modified at invoke time.

There have been discussions about invoke-time vs. build-time in terms of
specifying the driver behaviour - Can we discuss that a bit more?

Many people want to have a configuration file that can do almost
everything at invoke-time, but to cover all use cases that would
require turing completeness and a full-on scripting language, which
has been pooh-poohed already (and I don't agree with anyway).

My original suggestion was separating out "behavioural adaptation" and
"platform configuration" into different buckets.

"behavioural adaptation" could include the GCC compatibility story,
adding extra command line flags, exposing internal functionality etc.

"platform configuration" could include setting up a cross-toolchain,
multilib, include files, and even at a push changing the subtool to
invoke (for producing non-llvm output).

I'd suggest that behaviour changes should be done in C++ as it is
turing complete, fast, has precedent and these changes should in
general not differ between platforms or users.

Platform configuration can be done by a configuration file / spec
file, and because it is just setting parameters or options this can be
relatively simple, just variable assignment and string
manipulation/interpolation.

GCC compatibility is definitely an advantage, but if the driver infrastructure is tidied up then it becomes a lot easier to add new GCC-incompatible drivers as well. I think a lot of people would like to see a cl.exe-compatible one that can just be dropped into MSVC, and it might also be useful to provide a driver that had a less human-friendly set of options that is easier for an IDE to invoke (e.g. no defaults, everything explicitly configured and driven by the IDE's build description).

One other use case I forgot to mention: Objective-C runtime selection. This is easy on Darwin, because Apple ships an Objective-C runtime and supports it for the duration of that release. It's much harder on other platforms, where the runtime is a third party package and may support some variable subset of the features that clang supports. The driver currently sets a load of codegen flags based on the runtime on Darwin, and it would be great if we could have runtimes on other platforms just drop in a config file that clang would automatically pick up and use to describe them. This would also make my life easier when I add new features to the GNUstep runtime, as I could just flip a switch in the config file and have clang use them, rather than having to rely on users adding yet more flags to their make files...

David

-- Sent from my Apple II

The two can be combined I think - configuration for let say 90% of the cases + scripting language loaded as optional plugin for the rest.

Regarding configurations flexibility: Usually there are antagonism between the two polar structures of configuration trees: a) Structure symmetrical to full internal representations (fast and simple validation/loading/storing), z) Short, human-friendly DSL (possibly with inheritances, aliases, etc). In many projects the problem is resolved using additional tool which translates z) to a) on demand (only on z change), using a) templates as base.

Kind Regards,
Alek

I wondered if some of the tablegen infrastructure couldn't be reused for this, so we could have a configuration file format that could be compiled into C++ for the common cases, avoiding the need to parse it on every invocation.

David

-- Sent from my PDP-11

There's no need to separate "90%" from "the rest" and design new DSL.
You can just use Lua as configuration file format.

“To GCC or not to GCC, that is the question”

+Doug Gregor, Miles Bader
-Sean Hunt, Christopher Jefferson, Andrew Trick

There is differing opinion on the amount of GCC compatability clang
should offer. A lot of examples and arguments have been given, but I
see one conclusion - the driver should not be pinned to anything.

Depending on policy decisions, the user interface should be able to be
changed with minimal effort.

GCC compatibility is definitely an advantage, but if the driver infrastructure is tidied up then it becomes a lot easier to add new GCC-incompatible drivers as well. I think a lot of people would like to see a cl.exe-compatible one that can just be dropped into MSVC, and it might also be useful to provide a driver that had a less human-friendly set of options that is easier for an IDE to invoke (e.g. no defaults, everything explicitly configured and driven by the IDE’s build description).

One other use case I forgot to mention: Objective-C runtime selection. This is easy on Darwin, because Apple ships an Objective-C runtime and supports it for the duration of that release. It’s much harder on other platforms, where the runtime is a third party package and may support some variable subset of the features that clang supports. The driver currently sets a load of codegen flags based on the runtime on Darwin, and it would be great if we could have runtimes on other platforms just drop in a config file that clang would automatically pick up and use to describe them. This would also make my life easier when I add new features to the GNUstep runtime, as I could just flip a switch in the config file and have clang use them, rather than having to rely on users adding yet more flags to their make files…

David

– Sent from my Apple II

I fully support this idea!

I think we would all benefit from a tidier architecture where we could have:

a clang driver: which gives us the opportunity to cleanly define option groups (whatever the policy, it can only be cleaner than gcc)
a gcc compatible driver: in which we struggle to maintain the compatibility with gcc interface as much as possible
possibly other special purpose drivers (MSVC is definitely one option, seeing as there is ongoing work to get MSVC compatible codegen and a Windows enabled libc++)

Note for compatibility drivers (gcc, MSVC): it’s unnecessary to bring new options here, as the purpose is to provide a compatible interface (drop-in replacement) not an actual improved one. Also, if we have the translation going, we might as well generate a small tool that takes a “compatible” command-line and spits out the equivalent clang command line.

And this “multi-entry points” approach actually allow us to contend with the presence of a configuration file as well: want to use a configuration file ? Invoke the “configuration-file” driver.
(Although honestly, we could perfectly provide the configuration file thing as a variety of scripts parsing the file and generating the suitable command line)

If we have a clean Driver design, then adding several flavors of options parsing should be easy. Of course we would still need be reasonable and not spew dozens of them…

– Matthieu