Proposal to update minimum Python version required to 3.8.
What version do you suggest to increase to?
3.8 (3.7 is EOL in ~5 months)
Why this upgrade?
Existing minimum has been EOL since Dec 2021.
Where is the pain in keeping the current min version?
LLVM devs are unable to test with the current minimum version (e.g., no longer carried by some distros or homebrew) and code that passes on newer version fails on the older due to (among others) changes in type specification.
What (common) dists / OSes would be affected (not have access to this version by default)
Finding these have been a bit difficult, pkgs.org only shows openSUSE 15.04, ALT Linux P9, Amazon Linux 2, Amazon Linux 2, Rocky Linux 8 as ones where 3.6 is available but not 3.8, but even for some of these spot check 3.8 was as easy to get installed as 3.6.
Centos 7 (Centos 8 seems to support)
Ubuntu 18.04 default Python install
RHEL 8 seems default is Python 2.7 and recommendation is to use SCL for Python 3 (which has 3.8)
openSUSE 15.04 (only in experimental, official in Tumbleweed)
What’s the process to if you don’t have the right version. For users and developers.
Installing a new version of Python is relatively easy and doesn’t require root access. Projects like pyenv, conda/miniconda/mamba make it easy to install additional Python versions. For many distros it is a case of just installing (e.g., for Ubuntu 18.04/mlir-nvidia buildbot one needs only apt-get pyton3.8 instead of python, CentOS 7 requires using SCL). Or just building from source.
We could start with updating the build bots and then set the minimum version in CMake.
It would also be worth codifying which components require python. What is there anything beyond python bindings? Ideally you don’t pay for what you don’t need.
I can think of LLDB which has python support (I don’t know if this is enabled by a flag?).
Lit uses (quite a bit) of python. Additionally, each individual project may have a nontrivial amount of project specific code - in particular, libcxx has quite a lot of test framework code of its own, running on top of lit. Libcxx also has some scripts uses for wrapping execution (libcxx/utils, run.py and ssh.py), where the main one, run.py gets used in most cases, while ssh.py only is used in remote testing setups (so a regular run through a normal test setup wouldn’t exercise that code).
Additionally, a bunch of scripts in llvm/utils (update_*_test_checks etc) also are written in Python.
Yes, Buildbot and I commented on a commit where it seemed like it would break too (but to test they or I have to fire up docker). The reason I mentioned the change in typing support is that has changed how many folks write new Python code.
@mstorsjo and @tschuett mentioned the others. lit is the one that I was thinking of being least easy to ignore (well unless you aren’t testing which I would consider very rare but have not data to back that). So we can require conditionally same as today only if you are testing/using lit, building LLDB or generating MLIR Python bindings (And I’d prefer that to be 1 version). We could expand the minimum requirement there to specify which components need it or potentially instead add “asserts” in the CMake file of those components to make the tracking more active.
In general, I think we should try to keep the requirements to build LLVM as low as possible. It’s already pretty complex to get started with as it is.
LLVM has always required fairly specific versions of C++ compiler, since that’s a core technology that the project uses to the max, but for auxiliary tools like Python I don’t think it makes sense to be as aggressive.
Perhaps sub-projects that are Python heavy (LLDB, MLIR?) could have stricter requirements, but for just llvm+clang+lit tests it would be nice if pretty much any Python 3 version was sufficient.