[RFC] Backend for Motorola 6800 series CPU (M68k)

Hi All,

Just a quick update on the Motorola 6800 backend: Based on the feedback, “M68k” (with lowercase “k”) will now be the canonical target name and “m68k” be the target triple name. I’ve updated all the patches under review to reflect this change.

I’m also asking for everyone’s help to review all the patches.
/* Target independent changes */

  1. https://reviews.llvm.org/D88385
  2. https://reviews.llvm.org/D88386
    /* M68k CodeGen */
  3. https://reviews.llvm.org/D88389
  4. https://reviews.llvm.org/D88390
  5. https://reviews.llvm.org/D88391
  6. https://reviews.llvm.org/D88392
    /* M68k Clang supports*/
  7. https://reviews.llvm.org/D88393
  8. https://reviews.llvm.org/D88394
    Especially the Clang part, so far no one has given it a pass.

We also had a simple roadmap here:
https://github.com/M680x0/M680x0-mono-repo/projects
Showing the prerequisites to become an experimental target and eventually, an official target. We’re currently struggling on setting up the buildbot but I believe Adrian (CC-ed) is working on that. So I hope the patches can be sorted out while waiting for the buildbot.

CC-ing Craig here since he might able to give some feedbacks

Thank you!
-Min

Hi Min!

Showing the prerequisites to become an experimental target and eventually,
an official target. We're currently struggling on setting up the buildbot
but I believe Adrian (CC-ed) is working on that. So I hope the patches can
be sorted out while waiting for the buildbot.

The m68k machine is actually already up and running, I just need to set up
a buildbot worker which I will do tomorrow or the day after tomorrow.

root@akiko:~# uname -a
Linux akiko 5.10.0-rc1-183617-gd7f4e16357f6 #1 Mon Nov 2 12:06:55 CET 2020 m68k GNU/Linux
root@akiko:~# cat /proc/cpuinfo
CPU: 68040
MMU: 68040
FPU: 68040
Clocking: 1085.6MHz
BogoMips: 723.76
Calibration: 3618816 loops
root@akiko:~# free -h
              total used free shared buff/cache available
Mem: 3.2Gi 36Mi 3.1Gi 1.0Mi 53Mi 3.1Gi
Swap: 4.0Gi 0B 4.0Gi
root@akiko:~#

This new type of emulation was made by QEMU contributions of Laurent Vivier who is
going to give a talk on QEMU m68k this Friday at our next m68k meeting [1].

Adrian

Hello Min!

Showing the prerequisites to become an experimental target and eventually,
an official target. We're currently struggling on setting up the buildbot
but I believe Adrian (CC-ed) is working on that. So I hope the patches can
be sorted out while waiting for the buildbot.

The m68k machine is actually already up and running, I just need to set up
a buildbot worker which I will do tomorrow or the day after tomorrow.

I have requested the buildbot to be added now [1]. The worker is fully set up.

Adrian

Hello!

Just a quick update on the Motorola 6800 backend: Based on the feedback,
"M68k" (with lowercase "k") will now be the canonical target name and
"m68k" be the target triple name. I've updated all the patches under review
to reflect this change.

Are there any news on this? The M68k buildbot is ready to be enabled. It will
be enabled once the backend has been merged [1].

Adrian

As well as the actual patch reviews, has there been official approval that the M68k experimental backend can be added to trunk? I guess we need a "Backend: M68k" bugzilla component - is there anything else?

Sounds good. I'll file that bug tomorrow.

Thanks,
Adrian

I've looked at a few newer backends in LLVM and I think the policy is
unclear about how an experimental target is approved and reviewed.

* BPF [LLVMdev] [PATCH] [RFC v4] BPF backend
Tom said "it's up to Chris Lattner to approve new backends" I cannot
find discussions about the approval process around the time it was
committed (Jan 2015)
* ARC [llvm-dev] [RFC] Adding ARC backend
"but it would be great to have this in tree" may be vaguely counted as
an
* VE I cannot find related llvm-dev discussions around the time it was
committed (Jan 2020)
* CSKY (a stub has been added)
[llvm-dev] [RFC] Add a new backend called CSKY The
closest thing to an approval is the sentence "It is always exciting to
see new targets!"

Also worth noting that some of the initial review patches have just
one or no reviewer on the reviewer list (i.e. probably not get enough
scrutiny from community members).
(In addition, our Phabricator setting makes a patch green if anyone
accepts it - this makes it seem like the target is approved while the
reviewer probably just intended that "the patch is in a good shape".
Sometimes the author committed it in haste after accepted. Sometimes
this may be understood as the author might have waited for too long
before getting feedback)

I’ve looked at a few newer backends in LLVM and I think the policy is
unclear about how an experimental target is approved and reviewed.

Hi Fang-rui,

It’s true that there isn’t a definite rubber stamp procedure to approve things in LLVM. We have kept it that way for years because the stricter the rules the harder it is to act on (some definition of) common sense.

So we have documented what we expect of targets in the developer’s policy [1] and support policy [2]. It’s not easy to dig information on those approvals because we don’t have a formal process, it’s just emails on the list.

But in the past, and for all of those targets you mention, the “process” is to reach consensus. That’s done by having people voicing their support of the target on the list, and no one bringing substantial concerns.

I may have missed something, but I have not seen anyone raising any concerns on m68k, so I think we’re good on that side. I personally support the m68k target to be in LLVM and I think it would be a great addition to our list of targets.

During the patch reviews is when people are most likely to raise concerns, so before that happens, it wouldn’t be appropriate for anyone to “approve” the target, or that would create an issue if the code had too many issues with it.

So, once all patches are reviewed, all changes are in, all your local tests are green, and a number of people approve them, and there are no more comments, come back to the list (probably in a new thread) and do a final RFC, saying the code is good to go. You could repeat the basics, like who will be the code owner, the community behind, and who will create the infrastructure to test the experimental builds. That’s the thread that, if all responses are positive, you may start the merge of the code into master.

Also worth noting that some of the initial review patches have just
one or no reviewer on the reviewer list (i.e. probably not get enough
scrutiny from community members).

From what I saw, Craig, Roman and others were sending good, detailed reviews, so I refrained from comment.

Once everyone is happy and there aren’t enough reviewers, let us know and we’ll start pinging wider in the community.

(In addition, our Phabricator setting makes a patch green if anyone
accepts it - this makes it seem like the target is approved while the
reviewer probably just intended that “the patch is in a good shape”.
Sometimes the author committed it in haste after accepted. Sometimes
this may be understood as the author might have waited for too long
before getting feedback)

Right, this doesn’t seem to be clear on our review policy [3] (and may need some adjustment). We work with the assumption that people won’t unconditionally approve a patch if they don’t have confidence or experience in the area to do so. The policy states that it’s up to the reviewer to make their intentions known (please wait, ask others, etc). But authors, especially those that already do or will work closely with our community, should always ask themselves what is the right thing to do, and when in doubt, asking around (review, mail, irc, etc) is the best policy.

The expectation for bigger changes is to need multiple reviewers and approvals. New targets, policy updates, etc are the ones that need the largest number of approvals from “a wide representation in the community” (which is not well defined, I agree).

Also, if you are submitting a patch series, you should not commit any patch until the whole series is approved. This is to avoid half-baked changes going in, only to be reverted back due to some previously unknown big issue with a patch that wasn’t reviewed yet.

In your case, you may choose to commit your series in stages (pre, llvm, clang) for example, but you’ll still need to wait until the whole series is approved, by multiple people, and then strong support is voiced in the final RFC email to the list. This is special for new targets or very large changes.

From what you wrote, I think that’s what you were expecting, but the writing isn’t super clear, so I want to make sure there are no confusions: Don’t assume silence is approval.

When in doubt, ping people. Most of us are super busy and we can easily forget, so we don’t take pings as annoying, but as a necessary part of a highly distributed project.

Hope that helps.

cheers,
–renato

[1] http://llvm.org/docs/DeveloperPolicy.html#adding-a-new-target
[2] http://llvm.org/docs/SupportPolicy.html
[3] http://llvm.org/docs/CodeReview.html

PS: If after all reviews you’re still stuck and no one else is responding to your pings, feel free to contact me directly.

Hi,

To add to what Renato said:

I've looked at a few newer backends in LLVM and I think the policy is
unclear about how an experimental target is approved and reviewed.

It's generally good form to tag anyone who commented in the mailing list threads on any patches, especially people who objected. If they're happy in review, it's probably good to go.

Generally, the bar for being in-tree is fairly low, the bar to being removed from the experimental-back-ends list is much higher. An experimental back end is not built by default and is not in any of the binary releases.

Experimental back ends provide a probation period for the maintainer community. If bugs are fixed regularly, the back end makes good progress, the required infrastructure is maintained, there's engagement from the maintainers in reviewing other patches then it's easy to get the back end promoted to non-experimental. If not, then it will languish in the experimental state for a while and then eventually be removed (this will typically happen much faster if it requires any infrastructure in the generic CodeGen logic to support it and is imposing a maintenance cost on the rest of the compiler).

It sounds as if there's a fairly active community of m68k enthusiasts (I'm assuming that, in spite of the title of this thread, we are talking about the 68000, not the 6800), so I expect that the promotion to supported back end should go fairly smoothly.

Note that, so far, I've been about the only person reviewing the CSKY patches and have very little spare bandwidth to do it at the moment. I'd encourage the m68k maintainers to take a look at it and help out as good community members. All of the back ends in LLVM would be better if more people reviewed code for back ends other than the one or two that they work on.

David

Hello David!

Generally, the bar for being in-tree is fairly low, the bar to being removed
from the experimental-back-ends list is much higher. An experimental back end
is not built by default and is not in any of the binary releases.

Experimental back ends provide a probation period for the maintainer community.
If bugs are fixed regularly, the back end makes good progress, the required
infrastructure is maintained, there's engagement from the maintainers in reviewing
other patches then it's easy to get the back end promoted to non-experimental.
If not, then it will languish in the experimental state for a while and then
eventually be removed (this will typically happen much faster if it requires
any infrastructure in the generic CodeGen logic to support it and is imposing
a maintenance cost on the rest of the compiler).

Thanks for elaborating this and I think this is actually the way to go, for any
such project. It's a fair deal between both the upstream project and the new
maintainers.

It allows us to get a fair chance, on the other hand, it still gives LLVM upstream
the possibility to decide about the production readyness of the backend and consequently
whether it's worth to keep or not.

It sounds as if there's a fairly active community of m68k enthusiasts (I'm assuming
that, in spite of the title of this thread, we are talking about the 68000, not the 6800),
so I expect that the promotion to supported back end should go fairly smoothly.

Yes, despite its age, the architecture is still very popular. It has only little commercial
relevance, unsurprisingly, but there is still new hardware being made and there are
even new variants of the CPU being developed by the community, notably the 68080.

For me personally, LLVM is an incredibly interesting and important project and I'm more
than thrilled that we are getting a chance to upstream our backend into LLVM's
repository.

I'm currently trying to ramp up my personal effort to contribute to LLVM and make sure
we're getting the backend supported on the targets we have in Debian.

I have already set up a buildbot worker for linux-sparc64 that is being used and another one
for linux-m68k is ready and waiting. I will later add one for MIPS (I've got a fairly
fast Loongson board) and one for 32-bit PowerPC.

Note that, so far, I've been about the only person reviewing the CSKY patches and have
very little spare bandwidth to do it at the moment. I'd encourage the m68k maintainers
to take a look at it and help out as good community members. All of the back ends in LLVM
would be better if more people reviewed code for back ends other than the one or two that
they work on.

I currently don't have the expertise to do so. But as I previously stated, I'm very interested
to help keep LLVM in shape on the backends that are being used in Debian, especially the
less popular ones as I know that ARM, PPC and X86 are very well maintained.

Thanks,
Adrian

Hi All,

Thank you for all the feedbacks!

@Adrian, thank you again for setting up the build bot

@MaskRay @Renato @David, thanks for clarifying the processing. I’m also glad that no one is standing against M68k. However I’m also aware that there are only few reviewers on the list. And that’s probably the main reason why those patches were stalling in the past few months. Therefore I think it’s a good idea to put everyone in this thread as reviewers for now. If you’re not confident on reviewing some of the patches, please feel free to ping others that might be interested in reviewing it.
And @David, thanks for letting us know the situation on the CSKY backend, I’ll take a look.

Best,
- Min

Hi Simon!

I guess we need a "Backend: M68k" bugzilla component - is there anything else?

I have filed a bug report for this now [1] since I know understood what "component"
was referring to ;-).

Adrian

Hi All,

Another major update on this topic: currently I’ve addressed most of the feedbacks on the Phabricator patches. (I think only one was left and that’s related to Windows, but since our own buildbots only run non-Windows system I think it’s fine for now)

More importantly, I’ve fixed rest of the test failures, so now except one XFAIL test case, which takes more time to fix it, M68k passes all of its tests now :slight_smile:

Personally, I think the M68k backend is in good shape to become an experimental LLVM backend.

Currently only Patch 7 (D88393) has Renato’s semi-LGTM, and he was also asking other people’s opinion and LGTM. It would be great if anyone can help reviewing and approving these patches.

The journey of getting M68k backend upstream has gone pretty far and again, I really appreciate feedbacks and helps from reviewers and everyone on this thread!

Best,
-Min

Hi Min!

Another major update on this topic: currently I’ve addressed most of the feedbacks
on the Phabricator patches. (I think only one was left and that’s related to Windows,
but since our own buildbots only run non-Windows system I think it’s fine for now)

More importantly, I’ve fixed rest of the test failures, so now except one XFAIL test
case, which takes more time to fix it, M68k passes all of its tests now :slight_smile:

Personally, I think the M68k backend is in good shape to become an experimental LLVM backend.

Currently only Patch 7 (D88393) has Renato’s semi-LGTM, and he was also asking other
people’s opinion and LGTM. It would be great if anyone can help reviewing and approving
these patches.

The journey of getting M68k backend upstream has gone pretty far and again, I really
appreciate feedbacks and helps from reviewers and everyone on this thread!

Congratulations on this important milestone and the fantastic work you delivered.

I would have never expected that this effort would ever get this far, so I'm more
than impressed and happy about the news.

The backend is also already actively being used in another project, namely a compiler
explorer instance which features various compilers for m68k including your LLVM
port [1].

And, secondly, Nick Desaulniers will be giving a short talk on December 18th, on our
monthly m68k online meeting [2] where he will be talking about compiling the Linux
kernel for m68k using your new LLVM backend :-).

Adrian

Hi All,

Another update on this topic: Currently three patches have been approved, thanks for all reviewers’ time and efforts :slight_smile:

Here are the remaining patches:

  • Patch 1/8 D88385: Got one LGTM, however I believe Jessica Clarke (jrtc27) (CC-ed here) is listed as blocker. Does anyone here know the best way to contact Jessica? Since the patch has been blocked for quite a long time and pinging @jrtc27 on Phab didn’t seem to work

  • Patch 3/8 D88389 (Basic .td files)

  • Patch 4/8 D88390 (MC layer)

  • Patch 5/8 D88391 (Target lowering)

  • Patch 6/8 D88392 (IR Tests)

Except Patch 6, which only has minor formatting issues to fix, I didn’t see any major objections or concerns on other patches at this time point.

In addition to the patch review, tomorrow Dec 18th 19:00 CET (10:00 AM PST), Nick Desaulniers is going to give a talk on compiling Linux Kernel for M68k using LLVM.
Please checkout http://m68k.info/ for more info about the live stream

Best,
-Min

Hi Min!

Another update on this topic: Currently three patches have been approved, thanks for all reviewers’ time and efforts :slight_smile:

I've read the reviews. Awesome progress.

Here are the remaining patches:
- Patch 1/8 D88385: Got one LGTM, however I believe Jessica Clarke (jrtc27) (CC-ed here) is listed as blocker.

     Does anyone here know the best way to contact Jessica? Since the patch has been blocked for quite a long time
     and pinging @jrtc27 on Phab didn’t seem to work

I pinged jrtc27 on IRC (OFTC) and will keep pinging until we get an answer.

- Patch 3/8 D88389 (Basic .td files)
- Patch 4/8 D88390 (MC layer)
- Patch 5/8 D88391 (Target lowering)
- Patch 6/8 D88392 (IR Tests)

Except Patch 6, which only has minor formatting issues to fix, I didn’t see any major
objections or concerns on other patches at this time point.

Sounds great.

In addition to the patch review, tomorrow Dec 18th 19:00 CET (10:00 AM PST), Nick Desaulniers
is going to give a talk on compiling Linux Kernel for M68k using LLVM.
Please checkout http://m68k.info/ for more info about the live stream

Yep. I'm very exciting for this talk! And we're planning to put it on Youtube later.

Adrian

Hi All,

Happy New Year!

This is another update on the M68k backend.

Currently we still have 3 patches to review:

  1. (1/8) TableGen changes https://reviews.llvm.org/D88385
  2. (4/8) MC layer changes https://reviews.llvm.org/D88390
  3. (5/8) Target Lowering https://reviews.llvm.org/D88391

4th (D88390) and 5th (D88391) patches only have minor formatting issues.

The biggest challenge now is in the first patch, D88385. Currently the patch proposed to add a new TableGen backend (CodeBeads) to generate auxiliary data structures for variable length encoding as well as complex addressing mode encodings. This is equivalent to X86’s usage of MCInstrDesc::TSFlags to carry their instruction encoding. However, TSFlags only has 64 bits, where M68k’s current design needs at least 24 bytes to encode all the instructions. While we might able to find an optimal way to fit M68k’s info into 64 bits in the future, the refactoring efforts to do it now is huge, especially on changing instructions definitions in TG files.

Jessica, who is listed in the blocker list of this patch, raised concerns about the necessity of this specialized TG backend, and proposed to re-use X86’s way to handle this problem. That is, put the M68k’s encoding info into MCInstrDesc with modifications in the InstrInfoEmitter TG backend.

Simon thought special handling in the InstrInfoEmitter/MCInstrDesc for M68k doesn’t have a huge difference with adding a specialized CodeBeads backend like the patch is doing now.

Renado preferred to keep the CodeBeads backend since it works well for M68k and later refactoring, if there is any, will probably only affect M68k. He also thought “where” to put this implementation - whether a TableGen or MCInstrDesc change - should come before asking “why” doing this change.

Personally, I also prefer to keep the current CodeBeads implementation and optimize it in the future. Since:

  1. CodeBeads is an optional TG backend. It will not affect any existing Target.
  2. As mentioned earlier, it works well on M68k now. And if there is any changes on it or even removal on this TG backend in the future, the refactoring cost is pretty low.
  3. We don’t have many resources to conduct large scale fixes in a short time (the longer this patch series diverged from upstream, the higher the rebasing cost for other parts of M68k backend) on this issue right now.

I would happy to hear if there is any suggestion on solving this challenge.

Thank you for the all the reviewing efforts!

Best,
-Min

Hi All,

After nearly a year of efforts trying to bring the M68k LLVM backend upstream. I’m glad to see the goal is within our reach now :slight_smile:

Currently there are only two open patches: D88390 and D88391. They only received minor styling feedbacks in the past few days and I’ve already fixed them. I think they’re in good shape. After the release/12.x branch is cut (which is just couple of hours away) and all the patches are approved, I will commit all 9 patches to the ToT at once.

So I think this is good time to talk about some future plan about bringing M68k into an official target.

First of all, I would like to use bugzilla to track all the milestones in M68k target, in replacement of the Github Project (https://github.com/M680x0/M680x0-mono-repo/projects ) using now. In order to be more centralized and make it easier to notify people in this community for any updates. Currently there is an umbrella bug (Bug 48791) as the blocker of graduating M68k from experimental target. Any milestones will be depended by this bug. Feel free to link to this umbrella bug too if you feel a bug is crucial for M68k being a mature backend.

Second, I will collect more M68k toolchain use cases. On one hand they’re precious (compiler) test cases; on the other hand, we can get to know what are the most wanted features nowadays, in addition to those that are already in our mind.
Currently we already had connections with the M68k retro-computing community (m68k.info), Clang Built Linux (https://clangbuiltlinux.github.io/ ), and the project to bring Rust compiler — which uses LLVM under the hood — to M68k. Please spread the words if you know anyone that is interested in this topic!

These are basically my thoughts on the future plan. Last but not the least, this backend will never go this far without the helps from this community.

I especially want to shout out to people CC-ed in this email, they spent time reviewing the patches and discussing with us on how to make the backend better. Thank you very much!

Best,
-Min

Hi Min!

After nearly a year of efforts trying to bring the M68k LLVM backend upstream. I’m glad to see the goal is within our reach now :slight_smile:

Awesome news! Congrats and kudos for the very hatd work!

Currently there are only two open patches: D88390 and D88391. They only received minor styling feedbacks in the past few days and
I’ve already fixed them. I think they’re in good shape. After the `release/12.x` branch is cut (which is just couple of hours away)
and all the patches are approved, I will commit all 9 patches to the ToT at once.

That's really exciting to hear. I can't wait to spread the news once the code has been merged.

So I think this is good time to talk about some future plan about bringing M68k into an official target.

First of all, I would like to use bugzilla to track all the milestones in M68k target, in replacement of the Github Project
(https://github.com/M680x0/M680x0-mono-repo/projects ) using now. In
order to be more centralized and make it easier to notify people in this community for any updates. Currently there is an
umbrella bug (Bug 48791 <https://bugs.llvm.org/show_bug.cgi?id=48791>) as the blocker of graduating M68k from experimental
target. Any milestones will be depended by this bug. Feel free to link to this umbrella bug too if you feel a bug is crucial
for M68k being a mature backend.

That was my plan as well. I will create copies of the remaining issues from Github in Bugzilla once the 9 patches have been
merged.

Second, I will collect more M68k toolchain use cases. On one hand they’re precious (compiler) test cases; on the other hand,
we can get to know what are the most wanted features nowadays, in addition to those that are already in our mind.
Currently we already had connections with the M68k retro-computing community (m68k.info <http://m68k.info/>), Clang Built Linux (https://clangbuiltlinux.github.io/ ), and the project to bring Rust compiler — which uses
LLVM under the hood — to M68k. Please spread the words if you know anyone that is interested in this topic!

The NetBSD folks will certainly be interested. I'm CC'ing them in my answer. I'm very confident we will attract more developers
once the code has been merged. I'm very much looking forward to what the community will deliver. I know that people are very
enthusiastic in the community.

These are basically my thoughts on the future plan. Last but not the least, this backend will never go this far without the helps
from this community.

I especially want to shout out to people CC-ed in this email, they spent time reviewing the patches and discussing with us on how
to make the backend better. Thank you very much!

Thanks a lot for your work and don't forget to claim your Bounty once the code has been merged.

Also, let's discuss the Patreon later this week.

Adrian

Hi Min!

Currently there are only two open patches: D88390 and D88391. They only received minor
styling feedbacks in the past few days and I’ve already fixed them. I think they’re in
good shape. After the `release/12.x` branch is cut (which is just couple of hours away)
and all the patches are approved, I will commit all 9 patches to the ToT at once.

Seems the 12.x branch has happened now [1].

Adrian