RFC: Upstreaming fir-dev in small chunks of functionality

Hi all,

I've just posted a few patches to Phabricator to start a conversaton on
another way forward for upstreaming from fir-dev. What I'm trying to do
with these patches is upstream small chunks of codegen functionality
with the ability to end to end test the functionality being upstreamed.
To that end I've added here only enough code to codegen empty
subroutines and empty programs, and added lit tests for Fortran->FIR,
FIR->LLVM and Fortran->LLVM.

I've split the patches in 3 parts, one to add the relevant code to the
tco tool to allow us to lower subroutines to LLVM IR, one to add (a
stripped down version of) the bbc tool and one to add lowering from PFT
to fir. You can find these patches here:
https://reviews.llvm.org/D85508
https://reviews.llvm.org/D85509
https://reviews.llvm.org/D85510

These patches could then be built on with other patches adding
functionality in similar chunks (e.g. a possible next step would be to
add handling of functions with single return statements).

I've tried not to diverge from fir-dev much when moving code over,
however in some places I have omitted classes and hand-inlined their
functions where only 1 or 2 simple functions are called in the
subroutine code generation in order to keep the diff small. Eventually
(if we follow this path) when we come to add more functionality those
classes would be added back in.

What do people think of upstreaming in this way? Is this a possible way
for us to get the upstreaming process moving forward?

Thanks!
David Truby

Thanks David.

While I haven’t looked at your patches yet, I’m in favor of the direction that your approach takes. Specifically, I agree in terms of allowing for a more progressive path towards getting the code upstreamed and having a coordinated integration with testing plans that can proceed in parallel (vs. waiting for upstreaming to complete). If coordinated with testing pieces that should help from a code coverage perspective — I worry other paths are more difficult in terms of building the supporting testing infrastructure and also represent a challenge for people who could be helping to flush out the testing.

I’m not familiar with the nuances of the code here but this could also allow for more engagement across the community as the pieces are connected up in a more complete code generation path that would allow more effort towards integration of new features where codegen paths are currently blocked. For example, would this simplify aspects of getting OpenMP, OpenACC, etc. integrated with aspects of FIR in a similar fashion (smaller steps with enabled testing along the way, getting a better look at integration across the components, etc.)?

Thanks again,

—Pat

Hi David,

What's the benefit of forking llvm-project from fir-dev?

- Steve

    External email: Use caution opening links or attachments

    Thanks David.

    While I haven’t looked at your patches yet, I’m in favor of the direction that your approach takes. Specifically, I agree in terms of allowing for a more progressive path towards getting the code upstreamed and having a coordinated integration with testing plans that can proceed in parallel (vs. waiting for upstreaming to complete). If coordinated with testing pieces that should help from a code coverage perspective — I worry other paths are more difficult in terms of building the supporting testing infrastructure and also represent a challenge for people who could be helping to flush out the testing.

    I’m not familiar with the nuances of the code here but this could also allow for more engagement across the community as the pieces are connected up in a more complete code generation path that would allow more effort towards integration of new features where codegen paths are currently blocked. For example, would this simplify aspects of getting OpenMP, OpenACC, etc. integrated with aspects of FIR in a similar fashion (smaller steps with enabled testing along the way, getting a better look at integration across the components, etc.)?

    Thanks again,

    —Pat

Hi Steve

There is no benefit to disconnecting the fir-dev fork from llvm master. Quite the opposite. I think what we would all like to see is the fir-dev fork be fully upstreamed into flang. That way all development on llvm flang can take place on llvm master rather than in two separate places, as is the case today.

When committing code to llvm master it is beneficial to do so in incremental patches which deliver functionality that is well covered by testing. This is not how the fir-dev upstreaming is happening at the moment, as has been discussed at previous calls. David's patches propose a way to upstream fir-dev that preserves these qualities that we think is superior and there was no disagreement to that on the call yesterday.

The question is how could we proceed with this sort of approach?

Our discussion yesterday raised two points that we have to address:
  1. We must keep the fir-dev branch up to date with changes on llvm master so the two do not diverge and the fir-dev developers are not cut adrift from llvm master.
  2. We must maintain correct attribution on each patch that goes into llvm master if upstreamed in this way.

I think if we can address those concerns then we have a path forward with David's approach. Do you agree?

If so, I propose that David speak with Eric and talk over options for addressing these concerns. We discussed many potential solutions to the first problem on yesterday's call but I think it would be best for the developers concerned to make a plan they are comfortable with executing then get back to us.

To be clear, David is offering his time to this effort, including the work to keep fir-dev up to date and we want to find a way to do this that keeps everyone happy.

Thanks
Rich

Hi Steve

There is no benefit to disconnecting the fir-dev fork from llvm master. Quite the opposite. I think what we would all like to see is the fir-dev fork be fully upstreamed into flang. That way all development on llvm flang can take place on llvm master rather than in two separate places, as is the case today.

I don’t quite understand why the development of FIR (which is a critical component for flang right?) is not happening in-tree?

When committing code to llvm master it is beneficial to do so in incremental patches which deliver functionality that is well covered by testing. This is not how the fir-dev upstreaming is happening at the moment, as has been discussed at previous calls. David’s patches propose a way to upstream fir-dev that preserves these qualities that we think is superior and there was no disagreement to that on the call yesterday.

The question is how could we proceed with this sort of approach?

Our discussion yesterday raised two points that we have to address:

  1. We must keep the fir-dev branch up to date with changes on llvm master so the two do not diverge and the fir-dev developers are not cut adrift from llvm master.
  2. We must maintain correct attribution on each patch that goes into llvm master if upstreamed in this way.

I think if we can address those concerns then we have a path forward with David’s approach. Do you agree?

If so, I propose that David speak with Eric and talk over options for addressing these concerns. We discussed many potential solutions to the first problem on yesterday’s call but I think it would be best for the developers concerned to make a plan they are comfortable with executing then get back to us.

To be clear, David is offering his time to this effort, including the work to keep fir-dev up to date and we want to find a way to do this that keeps everyone happy.

I haven’t seen a followup to this since 3 weeks ago? Any update?

Thanks,

Hi,

Hi Medhi

We’ve had a rethink and we are going to wait for Eric and Jean to refactor the branch instead of trying to do it ourselves.

I’ll ask David to withdraw the patches so as not to block anyone else who might want to make progress here.

Looking forward to seeing the refactored fir-dev patches next month!

Ta

Rich