[RFC] Tensilica Xtensa (ESP32) backend

Hi All!

On behalf of Espressif Systems, I would like to propose Xtensa backend to be added as an experimental to LLVM project.

The new target should satisfy common rules (LLVM Developer Policy — LLVM 16.0.0git documentation) to be upstreamed in experimental mode.

Therefore, we prepared an overview about conformance of the Xtensa backend to these rules:

  1. Every target must have a code owner.
    I added myself in CODE_OWNERS.txt as responsible for Xtensa backend support in the first patch.

  2. There must be an active community behind the target.
    We have been developing the Xtensa backend project for 3 years, the latest version is ported to LLVM release 11.0.0.
    https://github.com/espressif/llvm-project . The Xtensa backend project now implements object code generation, architecture-dependent optimizations and it became possible to use clang to compile software projects for the EPSP32 / ESP8266 processor family.

  3. The code must be free of contentious issues.
    The Xtensa backend code deisgned with minimum changes in IR behaviour.

  4. The code conforms to all of the policies laid out in this developer policy document, including license, patent, and coding standards.
    We paid attention to all these requirements in code design.

  5. The target should have either reasonable documentation on how it works (ISA, ABI, etc.) or a publicly available simulator/hardware (either free or cheap enough) - preferably both.
    The Xtensa target is implemented in Qemu emulator qemu/target/xtensa at master · qemu/qemu · GitHub and has publicly available hardware implementations, for example ESP32/ESP8266 MCU’s family ESP32 Wi-Fi & Bluetooth MCU I Espressif Systems . We also created the Xtensa ISA documentation project https://github.com/espressif/xtensa-isa-doc , which is designed to simplify the patch review process, this project is based on public sources.

Some time ago, we already presented this project for discussion to the LLVM community in a letter
https://lists.llvm.org/pipermail/llvm-dev/2019-March/130796.html , and published a series of patches in the Phabricator:

  1. Recognize Xtensa in triple parsing code https://reviews.llvm.org/D64826 .

  2. Add Xtensa ELF definitions https://reviews.llvm.org/D64827 .

  3. Initial version of the Xtensa backend https://reviews.llvm.org/D64829 .

  4. Add basic * td files with Xtensa architecture description https://reviews.llvm.org/D64830 .

  5. Add Xtensa MCTargetDescr initial functionality https://reviews.llvm.org/D64831 .

  6. Add Xtensa basic assembler parser https://reviews.llvm.org/D64832 .

  7. Add Xtensa instruction printer https://reviews.llvm.org/D64833 .

  8. Add support of the Xtensa shift / load / store / move and processor control instructions. https://reviews.llvm.org/D64834 .

  9. Add basic support of Xtensa disassembler https://reviews.llvm.org/D64835 .

  10. Add relaxations and fixups. Add rest part of Xtensa Core Instructions. https://reviews.llvm.org/D64836 .

Currently, patches 1 and 3 have been approved, but other patches are still waiting for review. The patches 1-10 in the Phabricator are updated in accordance with the latest LLVM API changes and comments from the patches review.

Now exists a strong demand and interest from our large developer community for LLVM Xtensa backend and there are also some other companies in a world which have Xtensa based chips. For example, Xtensa backend helps to implement Rust for the Xtensa GitHub - esp-rs/rust: Rust for the xtensa architecture. Built in targets for the ESP32 and ESP8266 and TinyGo https://tinygo.org/faq/what-about-esp8266-esp32/ .

But for the further development of such projects, it would be much more convenient to have the Xtensa backend implementation in the main LLVM version.

So, it would be great if the LLVM community could help resume the review process.

All comments and suggestions are welcome!

Andrei Safronov

1 Like

Very late reply, but I would like to add that I’ve been using this Xtensa backend for quite some time in TinyGo and found it very reliable. The few bugs I’ve found have all been fixed in a reasonable time. To me, it seems like Espressif is committed to maintain this backend upstream.

The only reason I can image that could block the merging of this backend is that the official ISA documentation is closed. However, Espressif has gone out of their way to make it as publicly available as possible, even writing their own documentation based on public information in GCC and binutils.

I would love to see this backend merged. I am somewhat familiar with LLVM backend so I might be able to review some of it, but would like to see some more feedback on whether this backend should be merged or not.

(Sidenote: I think this thread should be categorized under “Code generation”, not “Project Infrastructure”. But I don’t think I am able to move it).

Dear LLVM community,

I want to make a review of the current state of the Xtensa backend and ask community for help with review patches of this backend in Phabricator. The Espressif Systems still maintain this backend and we plan to continue this activity in future. There is strong interest from Espressif community to see upstreamed version of this backend, for example from Rust community, TinyGo community (many thanks to @aykevl for Xtensa backend review), etc.

Over the past year, we have made various improvements to the Xtensa backend and for new official releases of LLVM we’ve made rebased versions and published them. Currently we have version ported on release 15.0.0-rc1 and we are waiting for official release 15 to update this version and to publish it. We also permanently updates published patches in Phabricator according to reviewers comments and latest changes in upstream version. Also, to help reviewers we published branch, based on upstream version GitHub - espressif/llvm-project at main_xtensa . The patch number 2 was approved in Phabricator, many thanks to all contributors. Currently we have approved patches with numbers 1-3 and need help to continue reviewing process of the patch number 4 and to approve it.

We are looking for help with reviewing process of the last patches of the Xtensa backend in Phabricator from the broader LLVM community and hope for feedback.

Thanks to all!
Best regards,
Andrei Safronov

Hi Andrei,

I think this would be a good contribution and am willing to review these patches. I was the Xtensa gcc maintainer for many years (and am still emeritus), so I have some familiarity with the architecture, although it has been quite a while since I actively worked on it.

Also happy for others to chime in.

I have now gone over all ten patches for the experimental Xtensa backend, and I think it is ready for submission, with the changes I suggested. Others had reviewed it for style and such prior to my look so there wasn’t much to comment on there.

There is a lot of additional work that could be done (especially loop instructions and branch-target alignment), but I think this is a good start and will give opportunity for fixes later.

After it has been submitted, a good next step would be a clang patch with allows using the target-triple directly from clang, but that is a front-end change and not relevant here.

Any other thoughts?

Where does this stand? As we have heard no further objections, I think it should be committed.

As far as I see there are some comments to be fixed in the patches. @andreisfr Could you updates patches. I know that we fixed recent @Sterling-Augustine comments locally, so looks like we can publish those changes:

After that the first 10 patches will be ready for landing. I suppose that someone from LLVM maintainers should commit them.

Hi folks. I’d recommend starting a thread on LLVM project channel of discourse (you can include a link to this thread) for discussion about inclusion in the project. I don’t see any specific concerns, but that would be a good way to get visibility. Hopefully we can get a decision for you turned around in a week or so.

Here is the new thread: [RFC] Request for upstream Tensilica Xtensa (ESP32) backend