RFC: Adding a fortran mode to the clang driver for flang

Hi All,

I’m starting work on a driver and frontend for the new flang (the

recently renamed f18 - not to be confused with any previous project also

of the same name). This has implications for the clang subproject, which

I discuss below.

Preliminaries

The goal is to make the in-progress flang compiler usable as soon as

possible.

I am motivated to keep things simple and deliver soon, hopefully in

digestible chunks of work.

I have split this task into two smaller pieces in a top-down approach. First, a

driver needs to land which can invoke the flang compiler as part of a multi-file

compile+link when it is passed a Fortran input. Subsequently, the flang compiler

needs logic analogous to clang::CompilerInvocation, which takes a Fortran file

as input and produces output in the appropriate form for the command line

options specified.

This RFC is focused on the first part. There are implications for clang,

since there exists a significant body of driver code there, much of the

logic it would be beneficial to share and reuse. The first change will

be to clang. The flang changes will come in future RFCs, and I envisage

will be more an internal detail of the flang subproject. There will be

some leakage as we seek to share as much logic as makes sense to do so.

Approach

I assuming that flang intends to implement a gcc-like interface.

From https://clang.llvm.org/docs/DriverInternals.html:

[…] most of the driver functionality is kept in a library which can

be used to build other tools which want to implement or accept a gcc

like interface.

libclangDriver has the goal of being a reusable and extensible driver.

If it gained knowledge of the flang toolchain, implementing a flang

driver as powerful as the clang driver in terms of libclangDriver would

be a relatively small task.

I propose to put together a new “bin/flang” binary which would be

structured and behave in a similar manner as clang. The intent is to

mirror clang, for both the driver and CompilerInvocation, as much as

makes sense to do so. The aim is to avoid re-inventing the wheel and to

enable people who have worked with either the clang or flang

entrypoints, drivers, and frontends to easily understand the other.

The user-facing result will be:

When I do bin/flang -o foobar foobar.f90, the driver will invoke

bin/flang -fc1 foobar.f90 -o /tmp/foobar_cafe1234.o and subsequently

link it.

Preliminary prototype

https://reviews.llvm.org/D63607?id=206172 introduces the above

behaviour, to illustrate the approach. At this point I seek advice

primarily on the overall approach. If acceptable, I will tidy it up and

submit it for review. There will be more details to get right.

The patch is it stands has a small amount of logic which makes

bin/flang a symlink to clang, which causes the driver to go into

Fortran mode and invoke bin/flang -fc1; at the moment this is just a

stub which prints the arguments. In the real implementation, bin/flang

will be a standalone binary in the flang project. This hack enables

playing with my proposed driver approach now. At no point will flang

be required to build clang.

More details

bin/flang will naturally support mixed C/C++/Fortran inputs, invoke

the relevant compilers for each input and link everything together.

libclangDriver will gain a new “Fortran” mode (–driver-mode=fortran).

When in this mode, when given a Fortran input, the driver will invoke

the flang compiler, which would live at “flang -fc1” by analogy with

“clang -cc1”. “clang”, when not in --driver-mode=fortran, would retain

its existing default behaviour of falling back to gcc (and thus

gfortran), for now. This means that existing build processes relying on

the existing behaviour will not be touched, and that the new behaviour

is opt-in by either using bin/flang or --driver-mode=fortran.

bin/flang will use clang::Driver, in Fortran mode. This seems a source

of potential problems since it requires a clang::DiagnosticsEngine to

construct, and these things seem clang-specific. However, this

diagnostics engine is primarily used by the driver to print warnings

about user errors made in the options supplied, so maybe this is not a

big problem. I have not investigated the full implications of this yet.

The clang driver already has an option group for gfortran options.

libclangDriver will need to know about all of the options that the flang

frontend supports. The complete set is under development and will grow

over time. All options in this group would be passed through to the

flang compiler. It’s unclear to me at the moment whether the

gfortran-specific options will remain in their own group, or whether

they can be merged into a ‘fortran group’. This can be decided once the

full flang compiler options story is better understood.

Where it makes sense to do so, the flang compiler will accept many of

the same options that clang does, particularly those relating to the

output form (e.g. -emit-llvm, -S, optimization options, etc).

I would act under the assumption that where possible, the

non-passthrough logic in the driver should be minimized, and the heavy

lifting should occur inside the flang compiler. The compiler will reuse LLVM

option parsing logic where it makes sense to do so.

cmake and project structure: flang at the moment is an independent

project. Taking the above proposed approach will "grow a dependency

edge" from flang onto the clang subproject. This may be undesirable in

the long run. It implies that building flang will require building (at

least part of) clang. I think this should be addressed at some point so

that flang can be built independently, but perhaps not initially, as

moving the clang driver into another top-level subproject would be

another significant task. The approach and value for that task may be

clearer once a second dependency on libclangDriver exists under the

llvm-project tree.

Help text contamination: I’m operating under the belief that clang

should not mention the Fortran language options and likewise flang

should not mention the C-like-language options.

When in Fortran mode, the driver will need to pass additional options to

the linker, in particular to link the Fortran runtime and intrinsics.

I have not yet considered this in detail.

Implications

Building flang will require building libclangDriver, at least.

This approach gives a future path to having clang invoke flang for

Fortran inputs instead of falling back to gcc as it currently does,

giving better gcc drop-in compatibility without depending on having gcc

available.

Having this arrangement does mean that some changes - for example,

adding a new compiler option - will require patches which touch both the

clang and flang subprojects. In particular, Fortran language specific

options will continue to reside in clang/include/clang/Driver/Options.td.

Prior art

There is a lot of prior art in this domain. While considering the

problem, I have looked at the following projects:

This is a fork of clang, with the main goal of implementing the

toolchains/flang tool. The implementation that I propose in this RFC

is similar to that, but the tool would be simpler since the frontend

is starting from fresh.

Swift’s driver looks similar to that of clang, but is a separate

implementation.

  • llgo and other languages

These are written in other languages, and do not appear to reuse the

driver logic.

I observe that most prior implementations do not try to share logic with

clang. I surmise that the reasons for this are partly due to those

projects being external, and also due to those projects having quite

different user interfaces. Fortran on the other hand shares many

similarities with the C-like user interfaces, and there is a large body

of existing code relying on gcc-like behaviour implemented by the clang

driver.

Alternative approaches

The other approach I have considered is to do what Swift have done and

reimplement significant amounts of the driver, copying the structure

from clang. I don’t currently see a good reason to do this at this

stage, since flang is joining the LLVM project and therefore should be

able to share the clang implementation directly.

There may be other good reasons for doing this I’ve missed, in which

case I would be interested in hearing about it.

Future work and next steps

If there are no show-stoppers in my above proposal, I will submit the

attached patch for review, and commence further work on proposing,

prototyping and implementing bin/flang and bin/flang -fc1 in the

flang project.

All input appreciated.

Thanks for reading,

Peter Waller

peter.waller@arm.com

Arm Limited

I like the overall approach, especially as a starting point to get flang working.

I think that we should eliminate the dependence on Clang at some point by moving the driver code, and other shared parts (e.g., the OpenMP codegen), into llvm-core or a dedicated helper package. This will not only allow to build flang in a more standalone fashion but also to make it easier for new frontends to reuse all these parts.

Cheers,

Johannes

+1

                  -David

"Doerfert, Johannes via cfe-dev" <cfe-dev@lists.llvm.org> writes:

Hi, Peter,

I think that this is the right approach. First, Clang’s driver contains a lot of painstakingly-constructed code to find system libraries and link programs, including with sanitizers and other relevant things, on various targets, and aside from the fact that Fortran has some additional runtime libraries, the Fortran driver will need all of that logic. Second, mixed C/Fortran and C++/Fortran applications are very common, and so the driver which links a Fortran application should also know how to link in anything also relevant for C++ compilation. Third, Clang already has support for Fortran-specific options and knows how to compile Fortran code (by passing it to GCC’s driver), and extending this support for Fortran code (in a general sense) to support LLVM’s own Fortran compiler makes a lot of sense to me.

Hi All,

I’ve updated the differential revision for my implementation. PTAL:

Regards,

  • Peter