Adapting and open-sourcing PGI's Fortran frontend for LLVM

Hi everyone,

I have some very good news for everyone interested a production-quality Fortran frontend for LLVM:

The U.S. Department of Energy’s National Nuclear Security Administration and its three national labs have reached an agreement with NVIDIA's PGI division to adapt and open-source PGI's Fortran frontend, and associated Fortran runtime library, for contribution to the LLVM project. PGI is now working on isolating the necessary frontend components from their existing compiler code base and adapting those components to produce LLVM IR for consumption by our existing infrastructure.

We're expecting an initial source-code release from PGI by late 2016, and while that's still far away, PGI's developers will be working closely with the community to coordinate development efforts by others interested in Fortran support (e.g. on ABI support code for other platforms). Even before source code is available, PGI is going to keep the community updated on their progress. After source code is released, PGI will continue development work in coordination with the community to implement additional Fortran and OpenMP features not currently supported by PGI Fortran. The frontend will use LLVM's infrastructure for regression testing, and PGI will be developing regression tests for these new features in accordance with our existing practice.

I realize this is somewhat unconventional, but I'm confident that this approach will allow the LLVM community to capture the considerable Fortran expertise of PGI's team and provide our project with a production-quality LLVM frontend in a relatively short amount of time.

For the official announcement, please see:

https://www.llnl.gov/news/nnsa-national-labs-team-nvidia-develop-open-source-fortran-compiler-technology

-Hal

I won't arrive in time to ask @SC15 on Sunday, but some questions

Will this include their CAF runtime? What if any vendor extensions
will be supported? What about directives support like OpenACC/OpenMP?
Is the code itself written in C/C++ or something else? Which Fortran
standards will it support? Will they be open sourcing corresponding
tests?

From: "C Bergström" <cbergstrom@pathscale.com>
To: "Hal Finkel" <hfinkel@anl.gov>
Cc: "LLVM Dev" <llvm-dev@lists.llvm.org>, "flang-dev" <flang-dev@googlegroups.com>, "Rob Neely" <neely4@llnl.gov>,
"douglas miles (PGI)" <douglas.miles@pgroup.com>
Sent: Friday, November 13, 2015 4:32:16 PM
Subject: Re: [llvm-dev] Adapting and open-sourcing PGI's Fortran frontend for LLVM

I won't arrive in time to ask @SC15 on Sunday, but some questions

Will this include their CAF runtime? What if any vendor extensions
will be supported? What about directives support like OpenACC/OpenMP?
Is the code itself written in C/C++ or something else? Which Fortran
standards will it support? Will they be open sourcing corresponding
tests?

I'm going to defer to the PGI folks on answering these questions for now (except to note that, as stated below, there will be OpenMP support).

-Hal

What will happen to flang development efforts?

Alex

From: "Alex Bradbury" <asb@asbradbury.org>
To: "Hal Finkel" <hfinkel@anl.gov>
Cc: "LLVM Dev" <llvm-dev@lists.llvm.org>, "flang-dev" <flang-dev@googlegroups.com>, "Rob Neely" <neely4@llnl.gov>,
"douglas miles (PGI)" <douglas.miles@pgroup.com>
Sent: Saturday, November 14, 2015 1:41:08 AM
Subject: Re: [llvm-dev] Adapting and open-sourcing PGI's Fortran frontend for LLVM

> Hi everyone,
>
> I have some very good news for everyone interested a
> production-quality Fortran frontend for LLVM:
>
> The U.S. Department of Energy’s National Nuclear Security
> Administration and its three national labs have reached an
> agreement with NVIDIA's PGI division to adapt and open-source
> PGI's Fortran frontend, and associated Fortran runtime library,
> for contribution to the LLVM project. PGI is now working on
> isolating the necessary frontend components from their existing
> compiler code base and adapting those components to produce LLVM
> IR for consumption by our existing infrastructure.
>
> We're expecting an initial source-code release from PGI by late
> 2016, and while that's still far away, PGI's developers will be
> working closely with the community to coordinate development
> efforts by others interested in Fortran support (e.g. on ABI
> support code for other platforms). Even before source code is
> available, PGI is going to keep the community updated on their
> progress. After source code is released, PGI will continue
> development work in coordination with the community to implement
> additional Fortran and OpenMP features not currently supported by
> PGI Fortran. The frontend will use LLVM's infrastructure for
> regression testing, and PGI will be developing regression tests
> for these new features in accordance with our existing practice.

What will happen to flang development efforts?

This is obviously a somewhat-tricky question. There is an existing Flang code base and a few people have been working on it. Furthermore, it is still the only one we'll have for the next year. My hope is that, with the participation of the PGI developers, we'll be able to structure our Flang efforts on pieces that we'll be able to pick up and move to the code that PGI is releasing once that happens. Between development on the Driver, ABI support code, and testing and other infrastructure, the community can make substantial progress in 2016. By the end of 2016, we'll combine these pieces with PGI's code, and have a production-quality LLVM Fortran compiler representing the totality of this work.

-Hal

I hope Hal is correct, but more likely he's just giving a tactful and
politically correct answer.

Reality is more likely that flang will have better code and design.
We'll end up with a Fortran front-end that mirrors the current quality
and capability as PGI. Frankly and no offence meant, but I would have
been much happily if they had just sponsored development on flang and
or worked with NAG (who has a much stronger reputation for actual
Fortran conformance - I've never tested PGI tbh)

Keep in mind that PGI's Fortran fe is likely c and has been around for
a while.. (Since the 90's?) The PGI engineers are welcomed to placate
my curiosity and correct any wrong assumptions. I hope like heck that
we don't end up with actual Fortran code (in the FE or runtime). That
would introduce a circular build dependency which I can assure you is
a major pita.

Other curiosities - will the module format be compatible with PGI
compiler? What about F2K8 and future standards? Arrays? Will they
(when?) publicly document how they are mapping things in Fortran to
llvm ir or we'll just end up with a code drop? (I personally know of a
few challenges with mapping Fortran to llvm ir)

Will we have to do a public information request or how much is PGI
being paid for this..

+1 to Hal’s note regarding. I’m personally in full support of PGI efforts (I am not an employee of NVIDIA or PGI). I believe this is a great step for the future of Fortran as associated with LLVM. I hope to start the discussions with the PGI folks this week at Supercomputing. It should be an interesting year in development!

Here are answers to some of these questions, and a few others that haven't been asked (yet).

Doug

Will it support OpenMP? Our plan-of-record includes full front-end support for OpenMP 4.x; OpenMP for multicore CPUs built on top of the LLVM KMPC runtime should fall out pretty early; it is too early to say when OpenMP 4.x accelerator features will be supported, but we already have outside parties who have volunteered to help with this.

Will it support OpenACC? The hooks for OpenACC will be in the front-end, but we don’t have current plans to implement full OpenACC functionality in the LLVM infrastructure.

Will it support CUDA Fortran? We don’t have current plans to implement CUDA Fortran in the LLVM infrastructure.

What version of the Fortran standard is currently supported in the PGI Fortran compiler? Full Fortran 2003, with some Fortran 2008 features. We do not yet support Co-arrays, the block construct, or sub-modules; these are high on our priority list with respect to implementation of the remainder of Fortran 2008.

What about vendor-specific extensions? The PGI Fortran compiler supports Cray pointers and a number of legacy extensions from DEC, IBM and other compilers. All of these extensions will be retained in the open source version, but we are not making any attempt to proactively catalog and add other de facto standard extensions.

What processors will it support? It’s important to remember this is a Fortran front-end and associated runtime libraries. The front-end will emit LLVM IR which can be compiled by LLVM for a number of targets. Most of the development work will likely occur on Linux/x86-64, but it will be straightforward to re-host and re-target the front-end to other targets.

What about support for ABIs beyond Linux/x86-64? The bridge from PGI’s internal representation to LLVM IR is designed to support multiple ABIs; support for certain additional ABIs will already be in place by the time the source code becomes available; support for others should be straightforward to add.

Is the PGI Fortran front-end written in C++? The PGI Fortran front-end is currently written in ANSI C; we are re-engineering it to conform to LLVM coding guidelines, and will do a significant amount of refactoring and re-writing along the way; it is still TBD whether any parts of the front-end will be re-written or augmented using C++.

Will the modules format be compatible with the PGI Fortran compiler? Array descriptors? Name mangling? Fortran input/output files? Yes, but we may change the format of some of these from that used in the current PGI Fortran compiler.

Is any of the source code of the PGI Fortran compiler actually written in Fortran? Some of the Fortran intrinsic math library routines and intrinsic modules are written in Fortran, and we have no current plans to change that.

Will there be binary versions of the compiler available before the source code release? If so, will I be able to dump the LLVM IR prior to invocation of the LLVM back-end? Yes and yes, but we don’t yet have a timeline. Stay tuned.

What about the existing Flang project? We have already had discussions with most of the contributors to the existing Flang project. We hope and expect to join forces in a unified effort.

Will you still offer NVIDA/PGI-branded compiler products? Yes. Eventually, the Fortran compiler in the software products we ship and support will use the open source front-end and runtime, augmented with value-add features for our customers. There will be a transition period through the initial development during which the two front-end and runtime source code bases will be distinct.

How can I help? We are working on a list of tasks that contributors can potentially work on prior to the open source phase of the project. Stay tuned.

This sounds very interesting Hal (and Doug), and I’m glad to hear about it. I look forward to when more details and code is available,

-Chris

Hi

Just wanted to share with everybody my support to PGI efforts in Flang. I was happy to learn that we are going to fast-forward Flang to PGI’s level!

Thanks!

– Carlo

Hi Hal,
I haven't been following this closely, but has there been any updates recently.

Regards,
  Chad

Hi Chad,

I can tell you that progress is being made on PGI's side; I'll let Doug/Rob provide more detailed updates.

-Hal

Hi Chad - We have a functional Fortran compiler, with the PGI front-end bridged directly to LLVM, all of our Fortran runtime libraries integrated, and the Clang driver adapted for use with the Fortran compiler. We're working with a few users at DOE who are trying to compile big applications with a binary version of the compiler. Work is ongoing to migrate the source code into an LLVM-style repository, build infrastructure, documentation and web pages. All of the compiler and runtime source is now built with Clang or the Fortran front-end with LLVM. Much work remains on source code refactoring and documentation; we expect that work to accelerate once all the infrastructure work is done.

Doug

All excellent news! Thanks for the update, Doug!

Chad, et al,

In addition to Doug’s excellent technical update, I’ll note that we are
starting to have some discussions on the DOE side with PGI about
establishing a more formal review team made up of some key LLVM
stakeholders to help smooth the way for a broader public rollout of the
Flang code base and eventual integration. We’ll probably rely on Hal and
others here to help us figure out who that group might be. My expectation
is that it would involve a few WebEx/telecons on an as-needed basis
(perhaps quarterly?) starting later this summer when code starts to move
out of PGI’s internal systems and into a private github space. Stay tuned
- we (both DOE and PGI) are keenly interested in doing what we can to make
this successful.

—Rob Neely, LLNL

It seems to me that this "let's do it with selected people behind close doors" does not play well with "we are seeking to have some integration in the open-source projects and its broaden community".
My own personal impression is that not conducting it in the open blurs the "message" around Flang direction and goals.

No closed doors intended here. Just a recognition that for something like
an initial review to be useful, we probably have to be a bit careful in
how many people we can reasonably involve before it could get unwieldy,
and trying to be respectful of people’s time if we can nail down 90% of
issues with a smaller group before going broader. I think we’d be fine
with opening up the WebEx presentations to whoever wanted to observe and
possibly chime in if it could be a managed process.

—Rob

On 5/26/16, 2:06 PM, "mehdi.amini@apple.com on behalf of Mehdi Amini"

This process is certainly only open to a select group, so
pragmatically it's closed. I can understand that it will certainly not
be an easy process once it's public due to the amount of code and
complexity.

Maybe someone can comment on a specific issue -

When we ported our Fortran front-end to target llvm, we found that
Fortran ENTRY doesn't map very well to llvm ir.

Can anyone who is actually working on this migration comment how they
plan to handle that?
ENTRY docs
http://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vn99/

ENTRY is I think officially deprecated, but dinosaurs won't nuke
anything, ever.. (It's still used in the wild as well... ugh)

/* btw for anyone actually reading this - I think it's odd that nobody
has emailed me to ask for help reviewing. I must have thought wrongly
that everyone around here knows we (PathScale) ported Fortran to LLVM
IR since 5 years.. */

From: "C Bergström" <cbergstrom@pathscale.com>
To: "Rob Neely" <neely4@llnl.gov>
Cc: "Mehdi Amini" <mehdi.amini@apple.com>, "Pat McCormick" <pat@lanl.gov>, llvm-dev@lists.llvm.org, "Mike Heroux"
<maherou@sandia.gov>, "flang-dev" <flang-dev@googlegroups.com>, "douglas miles (PGI)" <douglas.miles@pgroup.com>
Sent: Friday, May 27, 2016 4:21:16 AM
Subject: [flang-dev] Re: [llvm-dev] Adapting and open-sourcing PGI's Fortran frontend for LLVM

This process is certainly only open to a select group, so
pragmatically it's closed. I can understand that it will certainly
not
be an easy process once it's public due to the amount of code and
complexity.

Maybe someone can comment on a specific issue -

When we ported our Fortran front-end to target llvm, we found that
Fortran ENTRY doesn't map very well to llvm ir.

Can anyone who is actually working on this migration comment how they
plan to handle that?
ENTRY docs
http://docs.oracle.com/cd/E19957-01/805-4939/6j4m0vn99/

ENTRY is I think officially deprecated, but dinosaurs won't nuke
anything, ever.. (It's still used in the wild as well... ugh)

How do you handle it? Is there any reason we can't just generate one function body per entry point? Where these might call some internal implementation function which takes a hidden argument that feeds an initial jump table.

/* btw for anyone actually reading this - I think it's odd that
nobody
has emailed me to ask for help reviewing. I must have thought wrongly
that everyone around here knows we (PathScale) ported Fortran to LLVM
IR since 5 years.. */

Lot's of people are reading :wink: -- Thanks for volunteering! I don't think that we've really put together the external reviewers on DOE's side of things, but please do contact the PGI folks out-of-band about early access to provide feedback.

Thanks again,
Hal