Tl;Dr;
Tl;Dr;
------
We would like to propose a new set of diagnostics classes for the
clangDriver library. A proof-of-concept implementation is available
here: ⚙ D92025 PROOF-OF-CONCEPT: Add new diagnostics classes for clangDriver (more details below).WHY
As mentioned earlier on cfe-dev [1] [2], we are implementing the new
Flang driver. This driver is implemented in terms of clangDriver, which
is great in terms of code re-use across LLVM sub-projects! However, this
also means that:* Flang depends on Clang
* Clang contains logic that's only relevant to Flang (e.g. [3])Both of these shortcomings will go away once clangDriver becomes
independent of Clang. In this RFC we focus on one particular dependency
of clangDriver on Clang - *Clang's diagnostics classes*.DIAGNOSTIC CLASSES
clangDriver relies on Clang's powerful diagnostics classes (e.g.
Diagnostic, DiagnosticsEngine, DiagnosticConsumer, DiagnosticBuilder)
for its relatively basic command-line errors and warnings. These classes:* are tuned for complex frontend/language diagnostics (i.e. related to
parsing, semantic analysis or for clang-tidy checks)
Sorry for the slight distraction, but what's the plan for Flang's
diagnostics? Might it benefit from Clang's diagnostic infrastructure
for tracking source locations, etc?
No worries. We discussed this briefly in the past (see e.g. [1]). AFAIK, there are no plans to share diagnostics infra between Clang and Flang.
Diagnostics (and SourceLocation) in Clang are tuned for C-family languages. Generalising that code would be non-trivial. Personally I think that such code re-use would be great, but sadly we wouldn't be able to commit to such refactoring anytime soon.
[1] http://lists.llvm.org/pipermail/llvm-dev/2020-June/142024.html
-Andrzej
Hmm - given the answer here:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142048.html - I
wonder whether it's likely there should/would be more code reuse.
Sharing the whole integrated assembler between clang and flang? I
don't see further discussion on that branch of the thread engaging
with that idea.
Thank you for your reply. I appreciate that there are other important parts of the toolchain that we haven't discussed in much detail yet. For now we've been focusing on the driver. Are you suggesting that perhaps we shouldn't be sharing/re-using clangDriver? Or have a broader discussion about other infrastructure instead?
I'm just trying to understand whether we've missed some important step or dependency here. From what we've seen and discussed so far, it seems that making clangDriver independent of Clang would be beneficial for both Clang and Flang.
Btw, we continued the original discussion in this follow-up RFC:
* http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
-Andrzej
>>
>>
>>
>>>
>>> Sorry for the slight distraction, but what's the plan for Flang's
>>> diagnostics? Might it benefit from Clang's diagnostic infrastructure
>>> for tracking source locations, etc?
>>>
>>
>> No worries. We discussed this briefly in the past (see e.g. [1]). AFAIK,
>> there are no plans to share diagnostics infra between Clang and Flang.
>>
>> Diagnostics (and SourceLocation) in Clang are tuned for C-family
>> languages. Generalising that code would be non-trivial. Personally I
>> think that such code re-use would be great, but sadly we wouldn't be
>> able to commit to such refactoring anytime soon.
>>
>> [1] [llvm-dev] [cfe-dev] [RFC] Refactor Clang: move frontend/driver/diagnostics code to LLVM
>
> Hmm - given the answer here:
> [llvm-dev] [cfe-dev] [RFC] Refactor Clang: move frontend/driver/diagnostics code to LLVM - I
> wonder whether it's likely there should/would be more code reuse.
> Sharing the whole integrated assembler between clang and flang? I
> don't see further discussion on that branch of the thread engaging
> with that idea.
>Thank you for your reply. I appreciate that there are other important
parts of the toolchain that we haven't discussed in much detail yet. For
now we've been focusing on the driver. Are you suggesting that perhaps
we shouldn't be sharing/re-using clangDriver? Or have a broader
discussion about other infrastructure instead?
It sounded like Richard was saying in the other thread, that if flang
was going to share the integrated preprocessor - then the amount of
code shared would be quite large and maybe flang depending on clang
would be the right solution to that, if that's the direction we're
going in.
But I didn't see any discussion of if flang would share that, or if
not, why not. If it is going to - then the driver refactoring may be
unnecessary, because the goal of flang not depending on clang might
not be suitable in a future in which flang shares clang's
preprocessor.
Our goal has always been _just the driver_.
In the follow-up RFC [1], we refined our plan and tried to emphasise that our plan is to:
"Make libclangDriver independent of Clang".
Richard's reply to that RFC [2]: "(...) this seems like a good direction to me". Hopefully Richard (CC'ed) can confirm that we didn't misinterpret that 
I don't think that Flang should depend on Clang. Also, that's something that was explicitly requested in one of our early patches for the new Flang driver [4]. And indeed, we committed ourselves to removing this dependency on flang-dev [5]. As for cfe-dev, in both [1] and [3] we proposed to move any shared code (in particular, libClangDriver) out of Clang. So again, no dependency on Clang.
AFAIK, there are no plans to share the preprocessor and hence no discussion in that direction is happening. Obviously I can't speak for everyone on flang-dev, so I CC'ed that mailing list for better visibility.
Thank you for your feedback!
-Andrzej
[1] http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[2] http://lists.llvm.org/pipermail/cfe-dev/2020-August/066488.html
[3] http://lists.llvm.org/pipermail/llvm-dev/2020-June/141994.html
[4] https://reviews.llvm.org/D79092
[5] http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
FWIW, agreement on the rough plan to split clang driver from the rest of clang was certainly how I remember the previous thread ending, and it still seems like a reasonable idea to me.
Regarding sharing a preprocessor between flang and clang, I think there wasn't much discussion about it because no one has an interest in doing it.
Preprocessing is superficially similar but Fortran has a lot of differences that would make sharing with clang very difficult.
Some examples of things a Fortran preprocessor has to handle:
- fixed form vs. free form and directives that switch between them
- preprocessor directives in all-caps in fixed form
- source lines truncated at or padded to 72 or 132 columns
- Fortran line continuations
- Hollerith constants and edit descriptors
- Fortran expressions in #if and #elif
- Fortran INCLUDE lines
There are more details about Fortran preprocessing here:
https://github.com/llvm/llvm-project/blob/master/flang/docs/Preprocessing.md
Tim
External email: Use caution opening links or attachments
>
> It sounded like Richard was saying in the other thread, that if flang
> was going to share the integrated preprocessor - then the amount of
> code shared would be quite large and maybe flang depending on clang
> would be the right solution to that, if that's the direction we're
> going in.
Our goal has always been _just the driver_.
In the follow-up RFC [1], we refined our plan and tried to emphasise
that our plan is to:
"Make libclangDriver independent of Clang".
Richard's reply to that RFC [2]: "(...) this seems like a good direction
to me". Hopefully Richard (CC'ed) can confirm that we didn't
misinterpret that
I don't think that Flang should depend on Clang. Also, that's something
that was explicitly requested in one of our early patches for the new
Flang driver [4]. And indeed, we committed ourselves to removing this
dependency on flang-dev [5]. As for cfe-dev, in both [1] and [3] we
proposed to move any shared code (in particular, libClangDriver) out of
Clang. So again, no dependency on Clang.
AFAIK, there are no plans to share the preprocessor and hence no
discussion in that direction is happening. Obviously I can't speak for
everyone on flang-dev, so I CC'ed that mailing list for better visibility.
Thank you for your feedback!
-Andrzej
[1] http://lists.llvm.org/pipermail/cfe-dev/2020-July/066393.html
[2] http://lists.llvm.org/pipermail/cfe-dev/2020-August/066488.html
[3] http://lists.llvm.org/pipermail/llvm-dev/2020-June/141994.html
[4] https://reviews.llvm.org/D79092
[5] http://lists.llvm.org/pipermail/flang-dev/2020-July/000470.html
Thanks for the context and links (& for Tim's follow-up as well,
though I don't understand most of the terms/Fortran features he
mentioned) & apologies again for the diversion.
At least a rough glance at the patch makes sense to me within this context.