@pat thanks for the guidance and suggestions. To make sure I understand, when you mention placing " a ‘status notes’ document at the top-level of the Flang repo," I’m imagining a STATUS.md file with checkboxes in the flang/ subdirectory of llvm-project repository on GitHub. That would be helpful as long as it’s maintained and referenced prominently in flang/README.md and if the items on the checklist are grouped under release milestones. What I think would be more maintainable and provide a much richer experience would be a GitHub project board similar to Semantic Tests for Parallel Features project board. In keeping with the theme of your message, the name of the new project could be “First Release Candidate.” The project board could group GitHub issues into columns with headings such as “To Do,” “Under Development,” “In Code Review,” and “On LLVM main.” This provides a lot more functionality than just a status list. For example, a GitHub project can be set up such that
Every issue tagged with the project name automatically appears in the “To Do” column on the project board,
Every issue assigned to a developer automatically moves to the “Under Development” column on the project board,
Commit messages that contain text like “fixes #42” can automatically trigger the closing of the corresponding issue when the commit is pushed to the main branch, and
Every issue closed moves to the “On LLVM main” column on the project board.
Moreover, developer discussions can be tracked as comments on the issues. Seeing such discussions in the open is really important. The lack of visibility into what’s happening has caused some in the Fortran community to express pessimism publicly about seeing anything useful from LLVM Flang anytime soon. See this recent comment in the Fortran-lang Discourse. Most Fortran programmers don’t fully appreciate the monumental amount of work that has happened and continues to happen in developing LLVM Flang.
The approach that I’m suggesting invites a broader group of people into the discussion. Users might offer comments on the relative importance of certain optimizations or features that can help prioritize development. Some users might even roll up their sleeves and contribute code. On the OpenCoarrays project, we’ve seen this happen several times over the years when we received unsolicited bug fixes and new features from people we didn’t know prior to their contributions.
Lastly, the project board gives a rough sense of the percent completion based on the number of issues in each of the various columns. It’s not perfect because the list is likely to grow as new work items are discovered and not every issue requires the same amount of development time, but it’s still useful information for the potential user community in an open-source project. I hope these ideas are helpful.
IMO this is completely orthogonal to what the name and usability of the executable is during development.
My point is that at present, there’s more to it than that. If you are interested in joining development of Flang, you’d think that (and in my experience with other projects) it would be straightforward to try out the latest version; i.e. you clone the official repo, follow the build instructions, and get something potentially usable by a non-developer (i.e. you don’t need to know special names and hidden options). I don’t mean it won’t have missing features or bugs, but it shouldn’t present itself differently just because it’s not “done”.
Yes, and some manual testing from potential new contributors and users can assist in that endeavor. If you’re intentionally excluding them from early testing and feedback, you’re completely missing the bugs and features they might care about (or at least you can’t be sure).
Even with an alternative set of tests, if you have barriers to entry for potential new contributors or users (i.e. if you are not in “the club” and know “the secret handshake”), you will continue to struggle with these things.
Is there any plans to have a flang executable at all included with the 16 release? What are the chances of having one for the 17 release?
My main point has always been that these artificial barriers to early use are exclusionary, are preventing you from getting useful early feedback, and are turning potential contributors away. The sooner the official/upstream repository can stop this the better (IMO).
@rouson: My goal was to try and find a simple way to reach the community so there is a quick way to understand what is happening and where things stand in terms of a release candidate. Nothing less, nothing more. I’m hesitant to take on anything more complex until we can find a simple way to communicate these aspects so we can come to agreement on what the feature set for the first RC will be and do the work to bring it to reality.
@sscalpone Classic Flang has been completely useless across every project on which I’ve worked over the past ~5 years and that’s a long list of projects, some open-source and some not. The stagnation of Classic Flang is understandable because presumably it will be displaced by LLVM Flang eventually. Presenting a stagnant compiler that doesn’t support large parts of Fortran 2008 and 2018 to the community is not a good look for Fortran – particularly when there are already four compilers that are either 2018 compliant or or at least support the most significant and widely used parts of 2018. What would be the point of building interest in LLVM Flang by presenting the predecessor that led to a complete rewrite?
@pat the goal of the automations that I described are to save time. Teams I’ve led have used the described approach across several projects for years and I don’t recall a single team member suggesting that these tools were costing time. But the openness of the described approach is equally important. One of the main points of having GitHub issues is to foster discussion – preferably open discussion when the project is an open-source project. A lack of openness about the process is one cause of the impressions expressed in the Discourse thread that I cited earlier. A spreadsheet or checklist doesn’t open the dialogue and lacks a lot of functionality that would make life simpler.
@clementval thanks for the link. That project board is a great start. A board with the level of detail that is currently in spreadsheets would be really helpful – partly because it makes it more likely that users who visit more than once will see change over time. Also, if the cards listed on that project are converted to issues, then the issue comment threads can capture dialogue, including comments from potential users or potential contributors.
Stagnant aside, LLVM Flang doesn’t support large parts of Fortran 2008 and 2018 yet.
Classic Flang may not meet the needs of @rouson, however, it has a healthy development community, several commercial instantiations, and is available for FreeBSD.
How is this not deliberately misleading your future users? People will start using flang (as far as they know), and then when behavior drastically changes be completely confused. This is (IMO) the worst of both worlds. You’ll start getting bug reports and feature requests based on experiences with the thing you’re not actually working on. If the main argument against this renaming is not to inundate the developers with superfluous feedback, why on earth would you make this suggestion, which practically guarantees it?
We should listen to and support our community members who provide and maintain binary packages for LLVM Flang - ultimately they make sure that LLVM Flang reaches end users, which is vital for the success of the project. We should really avoid them having to maintain any downstream patches:
Also, can we please avoid including Classic Flang in this discussion? That’s a completely separate project.
This thread is an opportunity for the LLVM Flang community to make the life of binary package managers (and many other people) easier.
@everythingfunctional I moved my comment over here to continue the conversation because @tstellar suggested it was off topic in the [pitch] discussion.
The goal posts aren’t moving, but as you point out, the testing status is not publicly visible if we use the NAG suite as the measure.
@patsuggested a few open test suites and asked for suggestions.
The gfortran suite may be a good choice if you’d like to try it & report back.
Happily, the ball is still moving! There have been 31 commits to the llvm-project/flang tree this week!
A bit hesitant to chime in here given I don’t work on flang, but oh well…
The current state of LLVM flang - which I think is also called F18? - is really hard to figure out from the outside. I’ve been asked about this a couple times recently by users, and my honest answer has been that I just don’t know how mature it is. There’s been some great information in this thread, but summaries should really move into the documentation. From the sound of it, flang may not be quite ready for prime time, but what it does support and what reasonable user expectations should be at the current time really should be documented.
As for the naming point, it makes absolutely no sense to me to have a flang sub-project of LLVM whose binary is not named flang. Skimming (only!) the discussion above, it really seems like the discussion is commingling issues of naming with release management, documentation, and support. I’m very solidly in the “just call it flang” camp, and think this would be actively less confusing for users.
