Anyone that has been following the upstreaming discussion on llvm-dev will know that F18 has been under some (fair)
scrutiny with regards to its integration with LLVM and lack of use of LLVM data structures.
Richard Barton has asked me to look at this so I’ve made a cursory list of things that we should consider changing to
bring us closer to the rest of the LLVM project. If anyone has any additional suggestions please let me know.
If anyone wants to discuss any of these separately in more detail, feel free to start a new thread for discussion.
CMake Integration
- I believe this is already done by Eric from PGI, and is on github in his FIR pull request.
Can this be split off and merged separately to speed things up?
Change file handling to LLVM's file infrastructure
- Our current file handling infrastructure prevents building on non-Unix systems (e.g. Windows), and duplicating file handling across two sub-projects is unnecessary.
- LLVM's file handling works slightly differently to F18's current infrastructure, however I believe F18 doesn't use file handling in many places so once the necessary changes have been worked out once it should not take long to change it in every instance.
- A possible alternative would be to change F18's file handling to use LLVM internally but expose the same public interface. I consider this a less ideal but easier to implement solution.
Change uses of <iostream> to LLVM's streams
- For a number of reasons, LLVM has its own stream infrastructure to replace the standard streams.
LLVM's streams are a drop in replacement for standard streams (modulo type signatures) so this
should be a fairly simple change.
Remove Flang custom data structures that can be replaced with LLVM equivalents
- This part is more open ended, as there are a lot of LLVM data structures that could be used with varying
amounts of code change necessary.
- Below is a list of structures I have thought of so far:
• std::string → StringRef where appropriate
• EnumSet → llvm:PackedVector (not sure if this is directly comparable)
• std::vector → llvm::SmallVector where appropriate
• CharBuf → llvm::MemoryBuffer (not sure if this is directly comparable)
• std::set → llvm::SmallSet/llvm::StringSet/llvm::DenseSet where appropriate
• std::map → llvm::StringMap/llvm::DenseMap where appropriate
• std::list → Something else everywhere it's used. LLVM discourages the use of std::list as there's almost always a better choice!
Use llvm's error handling mechanisms
- Switch any unreachable cases to use llvm_unreachable with an error message. Does F18 have its own unreachable
Macro already? If so we can probably just search and replace for that.
- Use llvm::Error instead of error codes if and when error codes are used.
The CMake content in Eric’s PR is based on an early version that I wrote many months ago. I’m currently working on taking those components out of his request, reviewing it, I’m already addressing some minor details, cleaning up the organization a bit, double checking details relative to the broader practices across the community (e.g. Clang’s CMake config) and then I’ll finally do some initial testing (“in mono-repo” and “out of mono-repo” across a few platforms).
Once things seem OK I will re-issue just the CMake portion as a new pull request.
Anyone that has been following the upstreaming discussion on llvm-dev will know that F18 has been under some (fair)
scrutiny with regards to its integration with LLVM and lack of use of LLVM data structures.
Richard Barton has asked me to look at this so I’ve made a cursory list of things that we should consider changing to
bring us closer to the rest of the LLVM project. If anyone has any additional suggestions please let me know.
If anyone wants to discuss any of these separately in more detail, feel free to start a new thread for discussion.
CMake Integration
- I believe this is already done by Eric from PGI, and is on github in his FIR pull request.
Can this be split off and merged separately to speed things up?
Change file handling to LLVM's file infrastructure
- Our current file handling infrastructure prevents building on non-Unix systems (e.g. Windows), and duplicating file handling across two sub-projects is unnecessary.
- LLVM's file handling works slightly differently to F18's current infrastructure, however I believe F18 doesn't use file handling in many places so once the necessary changes have been worked out once it should not take long to change it in every instance.
- A possible alternative would be to change F18's file handling to use LLVM internally but expose the same public interface. I consider this a less ideal but easier to implement solution.
Change uses of <iostream> to LLVM's streams
- For a number of reasons, LLVM has its own stream infrastructure to replace the standard streams.
LLVM's streams are a drop in replacement for standard streams (modulo type signatures) so this
should be a fairly simple change.
Remove Flang custom data structures that can be replaced with LLVM equivalents
- This part is more open ended, as there are a lot of LLVM data structures that could be used with varying
amounts of code change necessary.
- Below is a list of structures I have thought of so far:
• std::string → StringRef where appropriate
• EnumSet → llvm:PackedVector (not sure if this is directly comparable)
• std::vector → llvm::SmallVector where appropriate
• CharBuf → llvm::MemoryBuffer (not sure if this is directly comparable)
• std::set → llvm::SmallSet/llvm::StringSet/llvm::DenseSet where appropriate
• std::map → llvm::StringMap/llvm::DenseMap where appropriate
• std::list → Something else everywhere it's used. LLVM discourages the use of std::list as there's almost always a better choice!
Maybe we should have it somewhere, e.g., on phab or GH so people can see
the current status.
If anyone wants to discuss any of these separately in more detail, feel free to start a new thread for discussion.
CMake Integration
- I believe this is already done by Eric from PGI, and is on github in his FIR pull request.
Can this be split off and merged separately to speed things up?
Change file handling to LLVM's file infrastructure
- Our current file handling infrastructure prevents building on non-Unix systems (e.g. Windows), and duplicating file handling across two sub-projects is unnecessary.
- LLVM's file handling works slightly differently to F18's current infrastructure, however I believe F18 doesn't use file handling in many places so once the necessary changes have been worked out once it should not take long to change it in every instance.
- A possible alternative would be to change F18's file handling to use LLVM internally but expose the same public interface. I consider this a less ideal but easier to implement solution.
Change uses of <iostream> to LLVM's streams
- For a number of reasons, LLVM has its own stream infrastructure to replace the standard streams.
LLVM's streams are a drop in replacement for standard streams (modulo type signatures) so this
should be a fairly simple change.
Remove Flang custom data structures that can be replaced with LLVM equivalents
- This part is more open ended, as there are a lot of LLVM data structures that could be used with varying
amounts of code change necessary.
- Below is a list of structures I have thought of so far:
• std::string → StringRef where appropriate
• EnumSet → llvm:PackedVector (not sure if this is directly comparable)
• std::vector → llvm::SmallVector where appropriate
• CharBuf → llvm::MemoryBuffer (not sure if this is directly comparable)
• std::set → llvm::SmallSet/llvm::StringSet/llvm::DenseSet where appropriate
• std::map → llvm::StringMap/llvm::DenseMap where appropriate
• std::list → Something else everywhere it's used. LLVM discourages the use of std::list as there's almost always a better choice!
Use llvm's error handling mechanisms
- Switch any unreachable cases to use llvm_unreachable with an error message. Does F18 have its own unreachable
Macro already? If so we can probably just search and replace for that.
- Use llvm::Error instead of error codes if and when error codes are used.
Please let me know if you think of anything else
(A lot of these have been listed elsewhere already)
Maybe we should have it somewhere, e.g., on phab or GH so people can see
the current status.
This sounds like a good idea, does Phabricator have a “projects” section where we can store this information and show which items are in progress? If so we should set this up on there so it is visible to the LLVM community at large rather than keeping it inside the F18 GitHub.
Somehow it completely slipped my mind that LLVM has a GitHub organization now!
GitHub has a “project board” feature that you can use to track things like this, you can see the one we’re using for tracking OpenMP semantic checking here: https://github.com/orgs/flang-compiler/projects/1
This might be a good way of tracking progress on this, what do you think?
I agree. we should have a flang project there anyway, tracking high level things there seems reasonable to me. Can you create such a page or so ppl need special permissions?
I think I could make the page if I had commit access to llvm, which I don’t. Do you know how I go about getting that or should I get someone else to do it?
David is getting the permissions to capture all this in a github project on llvm-project, meanwhile let me try and pull together the status of the upstreaming work.
Reminder of the goal: a set of changes that make F18 more LLVM-like before submission to monorepo and a plan towards achieving that. Further LLVMification can continue in the monorepo.