Minimum Python Version

I’d like to continue the discussion about minimum Python versions from the “Use multiprocessing instead of threading” thread in its own thread because I feel it warrants additional discussion.

In that thread, we were discussing maintaining support for Python 2.4 and 2.5. The latest response is:

My official opinion on Python version support is to maintain old compatibility, unless it is causing large problems with the code.

I would like to offer a counter opinion.

I believe LLVM should drop support for Python 2.4 and 2.5 for 2 main reasons:

  1. Python 2.4 and 2.5 are end-of-lifed
  2. Python 3 is coming

IMO, #1 should be reason enough. Let me explain #2.

All the Python in the tree will need to eventually support Python 3. “Modern” Linux distributions like Ubuntu are already shipping Python 3 as /usr/bin/python (while still offering 2.7 because the overwhelming amount of Python in the wild doesn’t yet work on Python 3).

The road to Python 3 for LLVM will likely come via Python code that works under both 2.x and 3.x (I’m assuming nobody wants a flag day). From my personal experience, I can unequivocally say that writing Python that works on both major versions is much easier the closer the Python 2 release is to 3. In other words, It’s much easier to dually support Python 2.7 and 3.x than it is 2.5 and 3.x. I feel the level of pain is pretty bad until you get to Python 2.6. Even then, there are dozens of small bugs in Python 2.6 and even earlier releases of Python 2.7 until 2.7.3 that make dual support difficult (especially in the area of Unicode handling).

For these reasons, I urge LLVM to drop support for Python older than 2.6. I would encourage requiring 2.7 (preferably the latest available release - 2.7.3 at this time) at the earliest convenience, but I’m not explicitly asking for it. While continued support for older Pythons is a noble goal and may continue to support people clinging to ancient Python releases, this will only make the path forward more difficult, as it puts an additional burden on those maintaining Python in the tree.

FWIW, I’ve been pushing this same argument at Mozilla for the Firefox tree [1], where so far it has been winning. We dropped Python 2.5 about 6 weeks ago and IIRC the only issue was we forgot to update one set of builders before we made the change. We have loosely agreed that we want to move everything to Python 2.7 (then 2.7/3.x and then eventually 3.x) and will be discussing this change in the days ahead. While I’m sure LLVM supports some more esoteric platforms than Firefox, I believe the data point is still relevant and LLVM could make a similar transition without a major headache.

[1] https://groups.google.com/d/topic/mozilla.dev.platform/djN02O03APc/discussion

Gregory

+1

I believe the need to support old versions is already causing problems
in the code base, the inability to switch to multiprocessing being the
most recent example.

Again, I realize that some platforms may have older versions of Python
pre-installed, but it shouldn't be a problem to install a newer
version of Python for the sake of LLVM development on such systems.

Eli

I'd like to continue the discussion about minimum Python versions from the "Use multiprocessing instead of threading" thread in its own thread because I feel it warrants additional discussion.

...

For these reasons, I urge LLVM to drop support for Python older than 2.6. I would encourage requiring 2.7 (preferably the latest available release - 2.7.3 at this time) at the earliest convenience, but I'm not explicitly asking for it. While continued support for older Pythons is a noble goal and may continue to support people clinging to ancient Python releases, this will only make the path forward more difficult, as it puts an additional burden on those maintaining Python in the tree.

That is all well and good, but please be reminded there are zillions of Red Hat (or CentOS) users out there, stuck with either Python 2.5 or 2.6, who cannot easily upgrade without busting their whole system...

To install a new Python version one doesn't have to "upgrade" and
surely not "bust" their whole system! You can install a newer version
of Python alongside older ones, and if everything else fails you can
just install it locally and use *that* to run the Python scripts LLVM
requires. It's quite easy to set up.

Eli

I'd like to echo how simple this is. Compiling Python from source is
literally configure + make. There are also tools like buildout [1] that
make it extremely easy to install multiple Python versions side-by-side.
And, since Python is prolific, you can bet that there exists an apt, yum,
etc package somewhere. I think simple instructions pointing to these would
be sufficient to not upset users of machines "stuck" on Python 2.5 and
below.

[1] https://github.com/collective/buildout.python

If you’re going to keep your version of LLVM up to date, I don’t see it being a stretch to require a reasonably current version of Python.

My official opinion on Python version support is to maintain old
compatibility, unless it is causing large problems with the code.

I would like to offer a counter opinion.

I think that we simply need to redefine "old". To me, an "old" version
of Python means 2.6. Something that is End-Of-Life'd like 2.{4,5} is
not "old", it is "dead".

I believe LLVM should drop support for Python 2.4 and 2.5 for 2 main
reasons:

1) Python 2.4 and 2.5 are end-of-lifed
2) Python 3 is coming

+1

-- Sean Silva

One of the most conservative distributions is Debian.

The python_defaults package has moved to 2.7.3 in Sid and 2.7.3~rc2-1 in Wheezy (Debian 7.0 now on its 4th beta and soon to be release candidate status).

I personally run Sid/Unstable in order to get general release builds of LLVM/Clang > 2.9, never mind 3.2.

It seems reasonable to target 2.7.3 as the oldest python release.

  • Marc J. Driftmeyer

The gcc compile farm currently only has python 2.4 and 2.5. I know Duncan is using it quiet extensively, especially all dragonegg buildbots run on it.

I very much agree we should ensure our python scripts are valid python 2.7 and as close as possible to python 3.x. However, as Daniel pointed out, there are still users of older python versions around. We could probably require them to upgrade, but I would like to avoid this, if we
can support older python versions without too much trouble.

Cheers
Tobi

Note that RedHat Enterprise Linux has _very_ long dates to end-of-extended-life-cycle

http://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux

RHEL4 has E-EOL of 2015, while RHEL5 has E-EOL of 2020. IIRC last time I checked Centos had a similarly long lifecycle.

This is just a piece of evidence about how long some distributions will be "supported". (I looked this up after I accidentally submitted a patch which used post Python-2.4 features that people pointed out broke stuff.)

Regards,
Dave

RHEL/CentOS is more conservative. RHEL 6 ships Python 2.6.6, RHEL 5 (which is still widely used) ships 2.4.3

This is an orthogonal issue. There is nothing saying the system Python version has to be 2.7+, just that a local version exists. Python is very simple to build on Linux. Certainly any admin of an RHEL pre-6 cluster can install a version somewhere.

Does the default GCC installation on RHEL 4 even build LLVM/Clang anymore, or do you need an “experimental” package of GCC 4.1 or a local build of 4.x?

RHEL4 has E-EOL of 2015, while RHEL5 has E-EOL of 2020. IIRC last time I checked Centos had a similarly long lifecycle.

In my opinion the burden of such ridiculously long cycles should fall
on RedHat and not all other OSS projects. They're presumably getting
paid precisely to handle the fact that the world moves on in 10 years.

</rant>

Tim.

Debian stable has 2.6.6. If you want to base which version of Python you use based on a particular OS's default (something others don't seem to agree with), then using _unstable_ as your baseline seems wrong.

This is a good point to consider. If RHEL 5 ships 2.4.3 and is
officially E-EOL in 2020, and we want to stick to "system Python"
everywhere, this means we plan to have our Python code running on
Python 2.4.3 until 2020? I doubt that anyone considers this seriously.

Eli

Extended support lifespans, probably not, but main support intervals, certainly tens of thousands of paying customers subscribe to Red Hat's model (or the hundreds of thoudsands usig CentOS rebuild of the same sources) take a stable API quite seriously

One thing I am missing here is:
   What is the ** NEED ** for chasing a later
   Python version?

-- Russ herrold

One thing I am missing here is:
        What is the ** NEED ** for chasing a later
        Python version?

Did you read Gregory's message that starts this thread?
Eli

I assume you mean the one in the archives:
   Sat Dec 1 14:57:48 CST 2012

Yes, actually, I did -- it boils down to: it's old, and Python 3 is coming

No current problems other than a speculative unicode issue are raised

-- Russ herrold

Yes, actually, I did -- it boils down to: it's old, and Python 3 is coming

Python 3.x is being made the default version of python on some systems, so getting python 2.x/3.x concurrent compatibility will probably be imminently needed. From experience, Python 2.4/Python 3.x concurrent compatibility is just plain impossible.

Also, I will point out that arguing that since RHEL 5 still uses 2.4 means we should keep our default at 2.4 is mildly specious, since running Clang on RHEL 5 has required, in my experience, several environment augmentations, most notably because the libc headers there won't compile in C99 mode, and I would be surprised if Clang is properly built by the default version of gcc there.

No current problems other than a speculative unicode issue are raised

I don't know any hard examples off the top of my head, but I do definitely remember (while grepping through python docs earlier today) being surprised that some of the functions I use on a consistent basis turned out to have a minimum of python 2.6.

Duncan, sorry for roping you into this thread, but it seems that your
bots are basically the only concrete need that has been voiced for
supporting End-of-life'd (2.4, 2.5) Python versions. Do you have any
plans for bringing those bots up to 2.6 or 2.7? If it wouldn't take
you a long time, I think it would be beneficial to update so that our
Python code can be made Python2+Python3 compatible; Arch Linux and I
believe Ubuntu 12.10 ship with Python3 as /usr/bin/python by default,
so being able to coexist with both is important.

-- Sean Silva