Hi All,
I’m starting work on a driver and frontend for the new flang (the
recently renamed f18 - not to be confused with any previous project also
of the same name). This has implications for the clang subproject, which
I discuss below.
Preliminaries
The goal is to make the in-progress flang compiler usable as soon as
possible.
I am motivated to keep things simple and deliver soon, hopefully in
digestible chunks of work.
I have split this task into two smaller pieces in a top-down approach. First, a
driver needs to land which can invoke the flang compiler as part of a multi-file
compile+link when it is passed a Fortran input. Subsequently, the flang compiler
needs logic analogous to clang::CompilerInvocation, which takes a Fortran file
as input and produces output in the appropriate form for the command line
options specified.
This RFC is focused on the first part. There are implications for clang,
since there exists a significant body of driver code there, much of the
logic it would be beneficial to share and reuse. The first change will
be to clang. The flang changes will come in future RFCs, and I envisage
will be more an internal detail of the flang subproject. There will be
some leakage as we seek to share as much logic as makes sense to do so.
Approach
I assuming that flang intends to implement a gcc-like interface.
From https://clang.llvm.org/docs/DriverInternals.html:
[…] most of the driver functionality is kept in a library which can
be used to build other tools which want to implement or accept a gcc
like interface.
libclangDriver has the goal of being a reusable and extensible driver.
If it gained knowledge of the flang toolchain, implementing a flang
driver as powerful as the clang driver in terms of libclangDriver would
be a relatively small task.
I propose to put together a new “bin/flang” binary which would be
structured and behave in a similar manner as clang. The intent is to
mirror clang, for both the driver and CompilerInvocation, as much as
makes sense to do so. The aim is to avoid re-inventing the wheel and to
enable people who have worked with either the clang or flang
entrypoints, drivers, and frontends to easily understand the other.
The user-facing result will be:
When I do bin/flang -o foobar foobar.f90, the driver will invoke
bin/flang -fc1 foobar.f90 -o /tmp/foobar_cafe1234.o and subsequently
link it.
Preliminary prototype
⚙ D63607 [clang][driver] Add basic --driver-mode=flang support for fortran introduces the above
behaviour, to illustrate the approach. At this point I seek advice
primarily on the overall approach. If acceptable, I will tidy it up and
submit it for review. There will be more details to get right.
The patch is it stands has a small amount of logic which makes
bin/flang a symlink to clang, which causes the driver to go into
Fortran mode and invoke bin/flang -fc1; at the moment this is just a
stub which prints the arguments. In the real implementation, bin/flang
will be a standalone binary in the flang project. This hack enables
playing with my proposed driver approach now. At no point will flang
be required to build clang.
More details
bin/flang will naturally support mixed C/C++/Fortran inputs, invoke
the relevant compilers for each input and link everything together.
libclangDriver will gain a new “Fortran” mode (–driver-mode=fortran).
When in this mode, when given a Fortran input, the driver will invoke
the flang compiler, which would live at “flang -fc1” by analogy with
“clang -cc1”. “clang”, when not in --driver-mode=fortran, would retain
its existing default behaviour of falling back to gcc (and thus
gfortran), for now. This means that existing build processes relying on
the existing behaviour will not be touched, and that the new behaviour
is opt-in by either using bin/flang or --driver-mode=fortran.
bin/flang will use clang::Driver, in Fortran mode. This seems a source
of potential problems since it requires a clang::DiagnosticsEngine to
construct, and these things seem clang-specific. However, this
diagnostics engine is primarily used by the driver to print warnings
about user errors made in the options supplied, so maybe this is not a
big problem. I have not investigated the full implications of this yet.
The clang driver already has an option group for gfortran options.
libclangDriver will need to know about all of the options that the flang
frontend supports. The complete set is under development and will grow
over time. All options in this group would be passed through to the
flang compiler. It’s unclear to me at the moment whether the
gfortran-specific options will remain in their own group, or whether
they can be merged into a ‘fortran group’. This can be decided once the
full flang compiler options story is better understood.
Where it makes sense to do so, the flang compiler will accept many of
the same options that clang does, particularly those relating to the
output form (e.g. -emit-llvm, -S, optimization options, etc).
I would act under the assumption that where possible, the
non-passthrough logic in the driver should be minimized, and the heavy
lifting should occur inside the flang compiler. The compiler will reuse LLVM
option parsing logic where it makes sense to do so.
cmake and project structure: flang at the moment is an independent
project. Taking the above proposed approach will "grow a dependency
edge" from flang onto the clang subproject. This may be undesirable in
the long run. It implies that building flang will require building (at
least part of) clang. I think this should be addressed at some point so
that flang can be built independently, but perhaps not initially, as
moving the clang driver into another top-level subproject would be
another significant task. The approach and value for that task may be
clearer once a second dependency on libclangDriver exists under the
llvm-project tree.
Help text contamination: I’m operating under the belief that clang
should not mention the Fortran language options and likewise flang
should not mention the C-like-language options.
When in Fortran mode, the driver will need to pass additional options to
the linker, in particular to link the Fortran runtime and intrinsics.
I have not yet considered this in detail.
Implications
Building flang will require building libclangDriver, at least.
This approach gives a future path to having clang invoke flang for
Fortran inputs instead of falling back to gcc as it currently does,
giving better gcc drop-in compatibility without depending on having gcc
available.
Having this arrangement does mean that some changes - for example,
adding a new compiler option - will require patches which touch both the
clang and flang subprojects. In particular, Fortran language specific
options will continue to reside in clang/include/clang/Driver/Options.td.
Prior art
There is a lot of prior art in this domain. While considering the
problem, I have looked at the following projects:
This is a fork of clang, with the main goal of implementing the
toolchains/flang tool. The implementation that I propose in this RFC
is similar to that, but the tool would be simpler since the frontend
is starting from fresh.
Swift’s driver looks similar to that of clang, but is a separate
implementation.
- llgo and other languages
These are written in other languages, and do not appear to reuse the
driver logic.
I observe that most prior implementations do not try to share logic with
clang. I surmise that the reasons for this are partly due to those
projects being external, and also due to those projects having quite
different user interfaces. Fortran on the other hand shares many
similarities with the C-like user interfaces, and there is a large body
of existing code relying on gcc-like behaviour implemented by the clang
driver.
Alternative approaches
The other approach I have considered is to do what Swift have done and
reimplement significant amounts of the driver, copying the structure
from clang. I don’t currently see a good reason to do this at this
stage, since flang is joining the LLVM project and therefore should be
able to share the clang implementation directly.
There may be other good reasons for doing this I’ve missed, in which
case I would be interested in hearing about it.
Future work and next steps
If there are no show-stoppers in my above proposal, I will submit the
attached patch for review, and commence further work on proposing,
prototyping and implementing bin/flang and bin/flang -fc1 in the
flang project.
All input appreciated.
Thanks for reading,
Peter Waller
Arm Limited