Not yet; we’re still implementing some important features in lowering and still cleaning up discrepancies in semantics. Doing this renaming now will lead to unhappy first-time users reporting bugs that we are already aware of.
What I heard from @rouson and @ktras was that in a recent weekly meeting they brought up this topic and heard that NVidia had completed the items they wanted done before removing their opposition. @sscalpone seemed to confirm this sentiment in the meeting this morning, albeit hesitantly.
@klausler, is there a publicly accessible list of items that need completed before you would remove your opposition? Are they things others could help with?
@everythingfunctional thanks for working on this and discussing it in this morning’s call.
@klausler several of us only have funding to work on LLVM flang through the end of the year and some of our funding is tied to user engagement. If we aren’t ready to encourage use, could you provide any insights on when you think we’ll be ready?
@clementval is still finishing up support in lowering for OOP and polymorphism. He’s doing incredibly fast high-quality work and I have no doubt that it will soon be largely complete. I have a couple of nontrivial open source codes in mind that use OOP/polymorphism, and I’d like to see them demonstrate correct execution before I consider this part done.
In semantics, I’m still working through a proprietary test suite’s error-catching tests, and am about 75% done with those; the non-error tests are clean. After finishing these error-catching tests, I’d like to get parsing and semantics into good shape with respect to open test suites, specifically GNU Fortran’s. And then get the LLVM bugs filed on GitHub against Fortran scrubbed and resolved. At that point, I’d feel ready to throw the doors open and get new bugs to work on.
That’s my list. Other people are working on runtime performance and OpenMP and OpenACC. I don’t have criteria for those, myself, other than not being embarrassed.
If people who want to contribute would like to find and fix problems exposed by open test suites or reported to LLVM’s bug tracker at GitHub, that would be most helpful. I bet that other vendors have licenses for proprietary test suites, too, and could use them to find/fix some of those remaining bugs. If more people were working on semantics now, I’d feel less trepidation about coping with bugs from new users later.
1 Like
Thanks, @klausler. This is very helpful. I had the impression that the goal was to complete Fortran 95 support before encouraging users to try LLVM Flang. Presumably I was mistaken because OOP and polymorphism entered the language in Fortran 2003 with some minor modifications in 2008 (e.g., intrinsic assignment to polymorphic variables). This underscores the value of a publicly available list. @sscalpone shared the link below to an Excel spreadsheet a while back, but it includes features all the way through 2018. Should we be looking for “OK” in the “Lowering status” column of that spreadsheet as a guide to what you would like to have complete before inviting users? If so, which rows? Or should we be looking at a different column or a different list altogether?
The bar I proposed before enabling flang to easily generate executables had three components. First, unimplemented features should present a meaningful message that referred back to the source code. Second, programs that compile ought to run as expected; the compiler should not force people to debug mis-compiled executables. Third, performance should be reasonably good without terrible outliers.
The first goal is in good shape for standard Fortran features. I don’t know if the OpenMP and OpenACC implementation have done the same. I expect that most Fortran codes will use one or the other. It is important to present the not-yet-implemented features in a good light.
The second goal is not fully implemented. There are semantically correct programs that execute incorrectly today which will be addressed as part of HLFIR. Also, there are issues with optimization, say at -O2, where bugs still exist. Fortran users expect optimizations to work.
The third goal is in reasonable shape but not finished. There are still some major outliers on standard benchmarks that will be addressed with the HLFIR work plus some optimization. Others are working on specific optimizations to bring flang inline with other compilers, e.g. inlining array intrinsics.
Who would be served by this change to flang? Not developers working on flang who can use what’s there today. It’s not onerous; it’s just cryptic. Not Fortran programmers who want to run their code. The “not yet implemented” messages are great, but they don’t run code. No one wants to debug runtime errors caused by mis-compilation, especially when the problems are known & fixes are in the pipeline. Good performance is expected from compiled Fortran programs.
The many contributors to flang continue to work hard to develop and deliver a great Fortran compiler. Making it easily available before it is ready has the potential to create negative feelings that will persist for years. Fortran developers have other free high-quality implementations available to them now.
I hope the community agrees & we delay this change.
1 Like
@sscalpone, @klausler and any others with oppositions, would it be a reasonable compromise to have this be a build time option? I.e. cmake SIMPLER_FLANG_USAGE=ON that defaults to OFF.
Who would benefit from such a change? Not the developers – we don’t need help yet finding things to complete or fix. Not the users – it would just make it easier to access what’s not likely yet to be a good first experience. What am I missing?
First, I want to say I absolutely appreciate that building a compiler takes a lot of difficult work, and I am grateful to everyone who has contributed. I hope to be able to contribute directly myself at some point.
That said, I disagree with the points made thus far.
First,
I don’t understand how that’s different from saying “We don’t want our users to tell us what’s important to them?” Not to say that what you’re working on isn’t important, but how do you know it’s the most important to your future users if you aren’t getting feedback? Maybe you’ll hear about lots of things you already know, but maybe it’ll turn out that more people would be happier sooner if you worked on this other thing first.
This whole “we don’t want users yet” sentiment is (IMO) antithetical to the ideals of agile software development and open source software in general. It’s contrary to three of the four tenants of the Agile Manifesto, namely:
Individuals and interactions over processes and tools
Customer collaboration over contract negotiation
Responding to change over following a plan
and actively avoiding the community aspect of the LLVM and flang communities.
Next,
Not making it easily available or being open about the process is already creating negative feelings. I see questions all the time about “I’d like to see how flang is doing. Can I try it out?”, “Well, yeah, but you have to compile it from source, and pass a special CMake flag to build it, and then the executable is called flang-new, and you have to pass a special, undocumented flag…” and at that point they give up. That’s a bad experience as evidenced by posts like:
I understand there’s an impulse not to release something until it’s ready, but that only really works for software that isn’t developed in the open. And even then it’s questionable. The first few paragraphs of this book tackle it head on.
And there is direct evidence that the opposite approach, being open and making it easy for people to try it out, in fact doesn’t lead to a bad experience. Case in point a “competing” project, LFortran. They’ve been very open from very early on and put considerable effort into making it easy for people to try it out as soon as possible. This has led to more people getting involved in the project and a considerable amount of understanding from early users, not a bunch of complaining and nobody coming back.
So
Everyone involved (IMO). You can start to rebuild trust and good will with your future users and the community. You will either find out you’re on the right track, in which case now you’ll know for sure, or you’ll find out you should change course in time to actually do something about it.
So please explain to me why you think it’s a bad idea to make flang easier for people to try out, besides just saying “it’s not ready yet” and then being a bit vague when answering the question “when will it be?”
4 Likes
Flang is a community effort. It’s hard to make predictions about the future in general, let alone when the fortunes of a project are based on the efforts of many individuals, companies, and organizations. I’m proud to be a member of the community and often happily surprised by new contributions and contributors.
@rouson, @everythingfunctional – Everything you need to build and distribute LLVM with a driver named flang is available in llvm-project. Go for it!
2 Likes
For the point discussed in this thread regarding Driver refactoring, we have not spent time on this. We plan to investigate the status later this month and provide an update.
This list shows the intrinsics only. A coarse-level features projects page is available in https://github.com/orgs/llvm/projects/12/views/1. Though it is not maintained well.
Issues from the gfortran testsuite are being added to https://github.com/orgs/llvm/projects/17.
This has not yet been done. We would like to reach OpenMP 1.1 feature completion before fixing all the issues from the testsuites. We are also enabling spec-2017 speed, spec-omp2012 and spec-hpc.
OpenMP 1.1 status can be found in OMP builder + Flang OMP tasks - Google Sheets. Other tabs in the same spreadsheet shows the status for other standards.
I hope you will be able to use the funding for improving flang.
Before the project moved fully to llvm-project and before the projects page in llvm-project became widely available, we created a list of applications (with build recipes for some) in Applications · GitHub. Due to change in priorities we could not continue maintaining it or move it to llvm-project.
A general issue the community has been facing is a lack of good open-source Fortran testsuites. We have to depend on proprietary suites and that makes it difficult to share the list of outstanding work.
Would you be able to help regarding the above two issues?
1 Like
I assume people have taken a look at gfortran’s test suite (both gfortran.dg and gfortran.fortran-torture, though the latter doesn’t look all that extensive)?
Yes. As mentioned above, the issues seen from the gfortran testsuites are being populated in https://github.com/orgs/llvm/projects/17/views/1. In the Fortran world there is one commercial compiler that is widely accepted to be most standard conformant and the testsuite of that compiler is the gold standard for checking conformance.
Somehow I managed to skim over that sentence… apologies.
@everythingfunctional , thanks for revisiting this!
I think that you’ve made some excellent points here and I agree with everything that you’ve said. I also feel that there are more constructive means of informing users that LLVM Flang remains “work-in-progress”, e.g. through
- up to date documentation,
- public presentations like the recent one by @kiranchandramohan : FOSDEM 2023 - Flang progress update, and
- bug trackers.
I think that keeping flang-new (instead of flang) in order to discourage users from using LLVM Flang is unwelcoming and unnecessary.
Btw, perhaps the renaming could be achieved in two steps:
- replace
flang-new -flang-experimental-execwithflang-new, - replace
flang-newwithflang?
Would that be an acceptable option?
-Andrzej
1 Like
Thanks for the words of support. I have nearly (I think) (re)constructed the changes necessary for step 2. I will work on the changes to accomplish step 1 as soon as I have confirmed. I had no desire to have a substantial delay between performing one versus the other, so explicitly doing them as two steps, in either order, did not seem to have a significant benefit, and would come at the cost of some people needing to modify their build systems twice instead of just once. I will save separate patches just in case though.
For those interested, I’ve officially submitted a patch to make this change:
Thanks for the patch, Brad.
The consensus is that this change is welcome, but it is premature to apply it. I’d like to postpone merging this change until we’ve reached consensus on when to do the renaming with enablement by default.
That said, your patch will help those who are building flang and wanting it to generate executables as a compiler named flang.
2 Likes
@sscalpone how do you come to the conclusion that there is a consensus that this is premature? @everythingfunctional has made strong arguments to the contrary and others (including at least @banach-space and me in this thread and others elsewhere) have concurred. How does that equal a consensus that this is premature?
1 Like
That’s not how I read this thread.
In general, I find it puzzling that folks were fine with a bash script called flang (that would call another, unrelated compiler), but reject a proper, well-tested driver to be called flang. Instead, the driver in LLVM Flang is stuck with some enigmatic name which was only introduced temporarily to avoid a name clash.
If there is no consensus for a big change just yet, then we should work towards a compromise. Together.
That was suggested last year as an acceptable alternative, see this comment. Since we disagree what the compiler driver in Flang should be called, this change would at least allow everyone to decide for themselves.
-Andrzej
1 Like