# Python code-style and reformatting, status update

**URL:** <https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641>\
**Category:** LLVM Project\
**Created:** [May 15, 2023, 9:13am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641 "2023-05-15T09:13:44Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 15, 2023, 9:13am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/1 "2023-05-15T09:13:44Z")

</div>

Hi everyone,

[As discussed earlier this year](https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style/68257), we want to enforce a code style for all the Python code in LLVM since we are racking up quite a bit of code that’s python these days.

At EuroLLVM last week, I was reminded about this effort and picked it up again, so I wanted to give a brief status update on this.

- Earlier today I landed the documentation of this here: [[docs] Add Python coding standard to documentation · llvm/llvm-project@83768e6 (github.com)](https://github.com/llvm/llvm-project/commit/83768e66108f583dcb2807a04662cfb9709ab2cf)
- I have also posted the first diff for mass-reformatting targeting the `llvm` subdirectory here: [⚙ D150545 [NFC][Py Reformat] Reformat python files in llvm](https://reviews.llvm.org/D150545)

Feel free to review or test if you see something weird.

As usual with mass reformats, it’s going to be a bit disruptive for people working in these files currently, but I hope it won’t be that bad, considering Python is more a util language in LLVM.

Next steps:

- Post more mass-reformating, I am planning to do this sub-dir, by sub-dir, except for LLDB, which @JDevlieghere wanted to handle since he is much more involved in that.
- Pre-commit CI step, to check the format. We want to run `black --check` on all `.py` files posted, but I am not sure it makes sense to work on this for Phabricator, considering we are planning to switch to GitHub “soon”. But I would like to hear input from @kwk @goncharov and @tstellar on this.

Thanks for your time!  
Tobias

---

<div class="post-metadata">

**Author:** ![wanders](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/wanders/32/2267_2.png) [@wanders](https://discourse.llvm.org/u/wanders)\
**Post date:** [May 15, 2023, 10:21am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/2 "2023-05-15T10:21:33Z")

</div>

> [@tobiashieta](#):
>
> Feel free to review or test if you see something weird.

I saw that your mass refactor contains changes to `llvm/bindings/python`. I was sure those were removed from tree.

There is a short thread on their removal here:

> [@Is anyone using still using the python bindings?](https://discourse.llvm.org/t/is-anyone-using-still-using-the-python-bindings/46063):
>
> Hi, The python bindings for LLVM haven't seen any meaningful changes in over 3 years. Is there anyone actively using these bindings? Can we consider removing them from the tree? -Tom

@tstellar Did you change your mind about removal? Should I go ahead with removal? They have now not seen much meaningful changes in like 9 years.

EDIT: Here patch for removal of llvm python bindings: [⚙ D150642 [bindings] Remove LLVM python bindings](https://reviews.llvm.org/D150642)

---

<div class="post-metadata">

**Author:** ![kwk](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/kwk/32/27451_2.png) [@kwk](https://discourse.llvm.org/u/kwk)\
**Post date:** [May 15, 2023, 2:43pm UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/3 "2023-05-15T14:43:22Z")

</div>

@tobiashieta having worked with Golang for quite a while I simply loved how the debates about style vanished from the picture. Black seems to aim at doing exactly that with Python. I’m all for it.

But I think that your question was more targeted towards whether or not we should put this on Phabricator, right? Well, I’d rather not put it into some existing check but instead have a distinct one just for python format checking. Otherwise it won’t get the attention it needs. Other test(s) simply fail to often and are therefore ignored.

What about using `precommit` git hooks for this? Then we don’t have to run any of this on the server as long as people install the hooks: [pre-commit](https://pre-commit.com/#2-add-a-pre-commit-configuration). This won’t prevent code from being gated but at least there’s a standard way how to get to proper formatted commits in the first place. I think both parts are needed: the pre-commit hook and the CI.

---

<div class="post-metadata">

**Author:** ![jh7370](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/jh7370/32/2079_2.png) [@jh7370](https://discourse.llvm.org/u/jh7370)\
**Post date:** [May 16, 2023, 9:03am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/4 "2023-05-16T09:03:01Z")

</div>

Have you considered what to do about lit tests that contain python snippets, often split out via the `split-file` util? These won’t have .py suffixes (though the test-time generated files might).

---

<div class="post-metadata">

**Author:** ![h-vetinari](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/h-vetinari/32/2121_2.png) [@h-vetinari](https://discourse.llvm.org/u/h-vetinari)\
**Post date:** [May 16, 2023, 9:31am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/5 "2023-05-16T09:31:11Z")

</div>

> [@tobiashieta](#):
>
> Earlier today I landed the documentation of this here: [[docs] Add Python coding standard to documentation · llvm/llvm-project@83768e6 (github.com) ](https://github.com/llvm/llvm-project/commit/83768e66108f583dcb2807a04662cfb9709ab2cf)

I was interested in what the lower bound is, given the last [thread](https://discourse.llvm.org/t/rfc-upgrading-llvms-minimum-required-python-version/67571/12) on this topic. The above-mentioned doc-change points to [GettingStarted.rst](https://github.com/llvm/llvm-project/blob/main/llvm/docs/GettingStarted.rst), which specifies `python >=3.6`, which was last [updated](https://github.com/llvm/llvm-project/commit/5e31e226b5b2b682607a6578ff5adb33daf4fe39) 2.5 years ago.

I’d suggest to adopt something like “the last Python version that’s still supported (by CPython)”, which would currently be 3.7 (until this [June](https://peps.python.org/pep-0537/#and-beyond-schedule)), and would give 5 years of support for any given version.

---

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 16, 2023, 9:33am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/6 "2023-05-16T09:33:10Z")

</div>

> [@jh7370](#):
>
> Have you considered what to do about lit tests that contain python snippets, often split out via the `split-file` util? These won’t have .py suffixes (though the test-time generated files might).

No, I haven’t - I was not aware of such files. Could you link me to one to see what they look like?

I don’t have a strong opinion on the matter; if someone that works with these files has, please let me know.

---

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 16, 2023, 9:34am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/7 "2023-05-16T09:34:02Z")

</div>

> [@h-vetinari](#):
>
> I’d suggest to adopt something like “the last Python version that’s still supported (by CPython)”, which would currently be 3.7 (until this [June](https://peps.python.org/pep-0537/#and-beyond-schedule)), and would give 5 years of support for any given version.

I think that makes sense, but it should be its own discussion. If I don’t remember incorrectly, there was already talk about bumping this in a recent thread.

---

<div class="post-metadata">

**Author:** ![jh7370](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/jh7370/32/2079_2.png) [@jh7370](https://discourse.llvm.org/u/jh7370)\
**Post date:** [May 16, 2023, 1:11pm UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/8 "2023-05-16T13:11:08Z")

</div>

A quick look and there are a few different styles. There are some files that use `%python -c` to run an inline snippet of python. I don’t think we should try to format these, as they are usually formatted weirdly anyway to ensure the python fits inline. Some tests (e.g. tools/llvm-objcopy/ELF/preserve-segment-contents.test`) create the python script at test time, so that it can contain references to lit substitutions. In the specific example, it writes these files using echo, and so I don’t think we’ll be able to get black to format this, even if we wanted to, so I wouldn’t bother.

I found very few examples (I thought there were more) using `split-file` and `%python`, so maybe it’s not a real issue. One concrete example is lld/test/ELF/lto/bitcode-wrapper.ll, which has a wrap\_bitcode.py “file” at the end of the test file. I suppose these should be formatted, but I don’t know if `black` can do partial file reformatting. I don’t think it’s worth trying too hard if it isn’t going to work.

---

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 17, 2023, 9:06am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/9 "2023-05-17T09:06:50Z")

</div>

## Update 23-05-17

llvm subdir reformatting has landed. With a few issues, I forgot to update the commit message telling people how to handle merge problems. It will be included in the future commits. I also forgot to format `lit.local.cfg` files that are all python. I have posted a follow-up below.

- ~~first llvm diff here: [⚙ D150545 [NFC][Py Reformat] Reformat python files in llvm](https://reviews.llvm.org/D150545)~~ Done!
- ~~llvm lit.local.cfg reformat diff here: [D150762](https://reviews.llvm.org/D150762)~~ Done!
- ~~clang and clang-tools-extra diff is here: [⚙ D150761 [NFC][Py Reformat] Reformat python files in clang and clang-tools-extra (llvm.org)](https://reviews.llvm.org/D150761)~~
- ~~libcxx/libcxxabi diff: [https://reviews.llvm.org/D150763](https://reviews.llvm.org/D150763)~~
- ~~mlir diff: [https://reviews.llvm.org/D150782](https://reviews.llvm.org/D150782)~~
- ~~The “rest” including bolt, lld and flang they where so few I bunched them together: [https://reviews.llvm.org/D150784](https://reviews.llvm.org/D150784)~~

Now I have reformatted all the files except lldb, I’ll leave that up to @JDevlieghere

---

<div class="post-metadata">

**Author:** ![kwk](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/kwk/32/27451_2.png) [@kwk](https://discourse.llvm.org/u/kwk)\
**Post date:** [May 17, 2023, 9:44am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/10 "2023-05-17T09:44:48Z")

</div>

@tobiashieta please, don’t forget to add the series of commits to `.git-blame-ignore-revs`.

---

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 25, 2023, 9:23am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/11 "2023-05-25T09:23:12Z")

</div>

Hi! Everything but MLIR and LLDB has now landed.

Any updates on the LLDB changes @JDevlieghere ?

---

<div class="post-metadata">

**Author:** ![JDevlieghere](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/jdevlieghere/32/2048_2.png) [@JDevlieghere](https://discourse.llvm.org/u/JDevlieghere)\
**Post date:** [May 25, 2023, 2:06pm UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/12 "2023-05-25T14:06:15Z")

</div>

> [@tobiashieta](#):
>
> Any updates on the LLDB changes @JDevlieghere ?

I was out of office, but plan to have it done before the end of the week.

Edit: [⚙ D151460 [NFC][Py Reformat] Reformat python files in lldb](https://reviews.llvm.org/D151460)

---

<div class="post-metadata">

**Author:** ![tobiashieta](https://sea1.discourse-cdn.com/flex021/user_avatar/discourse.llvm.org/tobiashieta/32/26681_2.png) [@tobiashieta](https://discourse.llvm.org/u/tobiashieta)\
**Post date:** [May 26, 2023, 6:10am UTC](https://discourse.llvm.org/t/python-code-style-and-reformatting-status-update/70641/13 "2023-05-26T06:10:08Z")

</div>

Hi! All open diffs have landed, and we have reformatted the whole tree’s .py files. We need to add some pre-commit ci checks, but I hope someone can step up and help with this. Otherwise, I hope reviewers can remind people to format with `black`.

Thanks for all the help landing this!