F18 ready to be merged + preview of merge

Hi llvm-dev

We believe we have completed enough of the agreed pre-upstreaming changes to start talking about merging F18 into LLVM. The live status is tracked at [1]. There are a few details that we have not managed to hammer out and we propose to tackle inside the LLVM monorepo. I have put a summary of these at the bottom of this mail.

Does anyone have any objections to flang being merged into LLVM with these issues still in-flight? Obviously we remain committed to solving them after merging into LLVM?

If that is all good then, as previously agreed, we want to give folks something more concrete to review and a bit of time to give feedback on it before we commit. David Truby has just now created a preview of what the LLVM project would look like with F18 merged in as flang. The obvious caveat is that both LLVM and F18 are continuing development so there will be additional commits on both sides of the real merge when it happens.

Here is the merge preview: https://github.com/DavidTruby/llvm-project/

This branch shows:

  • The commit history of F18 re-written as a straight line branch by Peter Waller’s script.
  • A commit that tweaks F18’s README to rename it as flang and make it more relevant inside the monorepo (being reviewed under https://github.com/flang-compiler/f18/pull/909).
  • The merge commit that adds F18 to the monorepo as flang
  • A patch to the monorepo cmake that adds flang as an optionally built target - see also https://reviews.llvm.org/D72416

I really encourage all folks maintaining buildbots or downstream builds to give this a look over to make sure it works for you. For everyone else, I hope this looks good to you too. All feedback very welcome.

If everyone is happy with that, we’ll agree on a new date in the regular F18 community call on Monday. I’ll be back in touch after that.

Thanks
Rich

Remaining Details still in-progress

[1] clang-format
F18’s clang-format file will have a few differences to the global formatting style. These are mainly ones that control alignment of code. We have not come to an agreement as a community on the best way forward here. This flang-dev thread summarises the debate. We would like to continue this debate after F18 becomes part of the monorepo.

[2] llvm_unreachable
Previously we stated that we would try to use llvm_unreachable in F18 whenever possible. Presently, F18 has a similar function called die, but this is used to cover multiple run-time error cases only some of which should be covered by llvm_unreachable. We would like to handle all cases together which also means coming up with a good system for reporting ICEs. See these code review comments for details. We propose to start fresh on this work after F18 becomes part of the monorepo.

[3] remove_if vs RemoveCarriageReturns
Whilst removing code that manipulated C-style strings we hit upon a case which seemed to show that std::remove_if was a regression on certain targets over the current algorithm using C standard memmove. This flang-dev thread summarises the situation and we are gathering further data before deciding what to do. We propose to continue to investigate this and make any necessary changes after F18 becomes part of the monorepo.

[4] Doxygen infrastructure
We are still working through the final comments on this patch and hope to have it merged before we become part of LLVM. As this patch simply adds the infrastructure to add doxygen comments and there are no doxygen comments in F18 source at present, we think that this should not block F18’s inclusion in the monorepo. We commit to finishing this off once F18 becomes part of the monorepo.

Created with Microsoft OneNote 2016.

Hi,

This sounds reasonable to me overall. There are some CMake issue to go through (I need to dig into it a bit).

When I run ninja check-flang at the moment (after fixing CMake) I end up with:

Testing Time: 16.22s

Hi Mehdi,

I can’t replicate those failures at my end, could you let me know what OS, compiler and CMake flags you’re using so I can try and reproduce?

Thanks!
David Truby

Hi Medhi

Definitely not expected, the tests should all pass.

As well as what David asked, the verbose output from lit would be helpful for debugging too.

Thanks for giving it a try.

Rich

Attached is the log.
I’m building with:

clang version 10.0.0 (https://github.com/llvm/llvm-project/ 3a6da1122b990386edeba0987d0d1fdc9c8dc53d)
Target: x86_64-unknown-linux-gnu
Thread model: posix

On some Ubuntu-like distribution.

I also ran with ASAN once and it found a bunch of leaks in bin/tco.

Best,

flang-fail.log (18.3 KB)

Hi Mehdi,

This looks like an issue somewhere in how CMake/lit is configuring the tests. Could you provide me with your CMake command line?

I’ll take a look at this first thing tomorrow but I need to know the CMake options used to reproduce it.

Thanks
David

Hi Mehdi,

This is the same list of failures that we were seeing on our FIR working branch. One problem seemed to be related to a misconfiguration in the cmake files as to where the system module files were located. The latest cmake changes seem to have got that straightened out. We have around 370 passing tests (0 failing) in our working branch.

A more recent run of the test tool, tco, with valgrind does not show any memory leaks. (We have had some bugs, of course.)

Hi Mehdi,

I’m unable to reproduce this issue right now. Are you ok with us going ahead with the merge (given that flang will not be built by default!) and fixing these build issues with a high priority post merge? The advantage of that will be that we will all be on the same commit of LLVM+flang, rather than trying to match commit versions across the two, so build issues will be much easier to debug!

Thanks
David Truby

Hi,

My cmake invocation was a simple as it can be and I invoked ninja check-flang (after building FileCheck explicitly since you were missing a dependency).
However it works if I run first ninja and then ninja check-flang, so it seems like the test target is missing dependencies.

This is not a blocker to me, it is perfectly fine to fix incrementally in tree.

Best,