[RFC] MLIR with respect to LLVM releases

Hi,

LLVM 17 release was branched earlier today.

At the moment the MLIR community does not qualify the LLVM release branch specifically, it is a snapshot of the MLIR development at the time of the release. I don’t think we are structured

However some people are still consuming it, and it is packaged by distributions. Also, for people who are using LLVM codegen with MLIR, depending on the latest stable release provides some stability and bugfixes.

I propose here 2 things that should be fairly low-overhead for us, but still provide some value to users consuming the releases right now:

  1. We should try to document changes from one release to the next. The refactoring and depreciation page is already one step, but I also just created a Release Notes page.
    I would want to encourage everyone to think during code review to suggest updating this document as we go. Going back on the last 6 months at the time of the release isn’t tractable.

  2. We should encourage MLIR contributors (in code reviews) to think about backporting bug-fixes in the release branch (that is for the weeks/months following a release branch creation).
    In particular any regression found in the next week are very likely good candidate for being cherry-picked in the release branch.

11 Likes

LLVM 17.0 is supposed to be final this Saturday, I think I backported 28 patches for MLIR in this release:

$ git log --pretty="format:%ad %h (%an): %s" --date=short  ^origin/main  origin/release/17.x -- mlir | cat -n
     1	2023-08-26 b66219d73500 (Markus Böck): [mlir] Fix infinite recursion in alias initializer
     2	2023-08-26 cc7e24c7a723 (Matthias Springer): [mlir] Fix crash when adding nested dialect extensions
     3	2023-08-25 94f348b7842a (Balaji V. Iyer): [mlir][math] Modify math.powf to handle negative bases.
     4	2023-08-25 ad5ed49a142b (Matthias Springer): [mlir][memref] Fix crash in SubViewReturnTypeCanonicalizer
     5	2023-08-23 5e47fe194595 (Mehdi Amini): Fix ODS verifier emission for DerivedAttr when Properties are enabled
     6	2023-08-08 a95298a0ed4b (Mehdi Amini): Fix canonicalizer to copy the entire GreedyRewriteConfig instead of selected fields
     7	2023-08-22 69946c8c9ce7 (Mehdi Amini): Fix MSAN error: use of unitialized value when hashing the MLIR pass manager (NFC)
     8	2023-08-22 c5f0c32da777 (Mehdi Amini): Fix MLIR pass manager initialization: hash the pass pipeline to detect when initialization is needed
     9	2023-08-22 0d8fd074b7a5 (Mehdi Amini): Fix some missing fully qualified namespaces in MLIR TableGen generator
    10	2023-08-20 67dca9da75b2 (Mehdi Amini): Fix MLIR build failure: error: no member named 'getValue' in 'mlir::OptionalParseResult'
    11	2023-08-09 5db0d770c7e8 (Mehdi Amini): Finish renaming getOperandSegmentSizeAttr() from `operand_segment_sizes` to `operandSegmentSizes`
    12	2023-08-05 3ac28602170b (Mehdi Amini): [MLIR] Make the `ConversionTarget` const ref in the DialectConversion (NFC)
    13	2023-08-05 600edc20b9dc (Mehdi Amini): Clarify the invariant of the MLIR pass pipeline around `Pass::initialize()`
    14	2023-08-04 45ad84a74230 (Benjamin Maxwell): [mlir][VectorOps] Fix folding of vector.extract from stretch vector.broadcast
    15	2023-08-03 13a8302b2e78 (Tobias Gysi): [mlir] Store segment sizes in std::array
    16	2023-08-02 6dc155bc91cc (Mehdi Amini): [MLIR][Bytecode] Add missing field initializer in constructor initializer list
    17	2023-08-01 416411fde507 (Valentin Clement): [mlir] Reduce warnings for bad assertion in generated code
    18	2023-07-28 6e2cf3856092 (Mogball): [mlir][llvm] Fix export of 64-bit integer function attributes
    19	2023-07-26 2c5ed2da4fb4 (Nicolas Vasilache): [mlir][gpu] NFC - Fail gracefully when type conversion fails instead of crashing
    20	2023-07-28 3e62997c4c0f (Thomas Raoux): [mlir] Fix arith verifier for tensor with encoding
    21	2023-07-28 6664b6e980fd (Alexis Engelke): [mlir][LLVMIR] Fix identified structs with same name
    22	2023-07-26 fc209e4158cc (River Riddle): [mlir-lsp] Guard writing output to JSONTransport with mutex
    23	2023-07-25 685bcc1d732a (River Riddle): [mlir:bytecode] Only visit the all regions path if the op has regions
    24	2023-07-24 99b39d7df623 (River Riddle): [mlir:bytecode] Support lazy loading dynamically isolated regions
    25	2023-07-24 45a4a1371b7e (River Riddle): [mlir:bytecode] Fix bytecode lazy loading for ops with multiple regions
    26	2023-07-25 6b3e6a9db7c3 (Mehdi Amini): Add release notes for MLIR
    27	2023-07-25 a50e3a66b991 (Oleg Shyshkov): [mlir] Fix assembly format parser generator after 9ea6b30ac20f8223fb6aeae853e5c73691850a8d.
    28	2023-07-25 f71f8c923ae9 (Jakub Kuderski): [mlir][spirv] Do not introduce vector<1xT> in UnifyAliasedResource

We can probably still squeeze critical crashes, otherwise future fixes will go in subsequent patch releases, the current schedule is the following (from llvm.org homepage):

  • Sep 5th: 17.0.0-final
  • Sep 19th: 17.0.1
  • Oct 3rd: 17.0.2
  • Oct 17th: 17.0.3
  • Oct 31th: 17.0.4
  • Nov 14th: 17.0.5
2 Likes

Thanks, Mehdi. What thought process did you go through in choosing patches to backport? Just trying to calibrate.

I focus on crashes and miscompile in general.

Sometimes some refactoring may be pulled in because of conflicts that makes it tedious to back port a change, it’s a judgement call depending on the risk and the time it would take to back port the bug fix without the refactoring.

Other cases like this:

2023-08-09 5db0d770c7e8 (Mehdi Amini): Finish renaming getOperandSegmentSizeAttr() from `operand_segment_sizes` to `operandSegmentSizes`

are more in the category of “usability fix”: I renamed most use cases right before the release branch was cut, and some downstream users alerted me that the inconsistency was surfacing for clients of the APIs, so I finished the renaming and backported the change as well.

Similarly for 2023-08-22 0d8fd074b7a5 (Mehdi Amini): Fix some missing fully qualified namespaces in MLIR TableGen generator ; which was a follow up to a change I made before the branch was cut, and would break some user trying to adopt properties if they aren’t in the mlir namespace.

I don’t remember why I picked 2023-08-05 3ac28602170b (Mehdi Amini): [MLIR] Make the ConversionTarget const ref in the DialectConversion (NFC) ; it is a very trivial patch but I don’t see how it was needed for one of the following fixes.

1 Like

Just a note that the 18 release branch was created: Release/18.x branch has been created

Let’s try to repeat the process from last time: if you’re authoring or reviewing a PR please think about whether it is a cherry-pick candidate!

1 Like