Different code in MLIR tools depending on `MLIR_INCLUDE_TESTS` (and fir-opt / `FLANG_INCLUDE_TESTS`)

MLIR
1

While working on packaging Flang in Gentoo, I’ve noticed that the fir-opt has some #ifdef FLANG_INCLUDE_TESTS that means that the tool’s behavior is slightly different depending on the value of this option. It was pointed out to me that MLIR suffers from the same problem, and indeed, I see that multiple MLIR tools have #ifdef MLIR_INCLUDE_TESTS conditions.

In my opinion, this goes against the principle of least surprise. It means that an option that’s described as responsible for “generating build targets for MLIR unit tests” actually changes how installed tools are built. This means that if you have two pipelines:

  1. that is intended to build MLIR, run tests, then install it.
  2. that is intended to build MLIR and install it, without running tests.

Then, if you set -DMLIR_INCLUDE_TESTS=OFF for the latter pipeline, you’re surprisingly going to get slightly different tools.

We’ve already discussed the same problem in Flang, but I think it would make sense to agree on the same solution for both MLIR and Flang, so I’d like to open a discussion here.

My “best” idea so far was to build a separate variant of the tools for tests. For example, mlir-opt would be always built without test-specific code, and if tests are enabled, an additional mlir-test-opt would be built with test-specific the code. Only the regular mlir-opt tool would be installed, and mlir-test-opt would be a build tree-only executable.

Throughout the discussion, it was also suggested that fir-opt isn’t really needed for end users, and that we may instead make it a test executable only, i.e. stop building and installing it otherwise. However, I don’t know if the same applies to MLIR tools involved, and given how long they have been installed, if there aren’t people actually using them.

What are your thoughts about the problem?

2 Likes
2

This has had a lot of litigation in the abstract arguing that point, but in my experience, users are quite clear: having access to these tools for MLIR based compilers without going all the way to a source build is strongly beneficial. It may not be for all users, but certainly quite a few (on every downstream I’ve worked on). It has also been argued in the past to simply include test passes/dialects in mlir-opt by default vs having all of the gating (I believe lattner has made this comment a couple of times but it’s old threads I can’t put my hands on right now).

A middle ground could be to gate *-opt individually on something like *_INCLUDE_DEV_TOOLS (or something named like that). If doing that, I would just unconditionally compile the test dialect/passes into it, leaving *_INCLUDE_TESTS to do what it says on the label: include the actual tests in the build system. You could use cmake_dependent_option to make sure that the dev tools are always built when tests are enabled.

Also strong +1 on not building tools differently based on the test define. I think that is the worst of possible situations, as you point out.

My 2 cents.

(Edit: removed a redundant paragraph)

1 Like