[RFC] RISC-V: a target for software (interpreted) rather than hardware (executed)

Code Generation RISCV
1

RISC-V is increasingly not only a hardware target, but also a software one. It gets decoded and dispatched by an interpreter, or handed to a simple JIT. For that kind of consumer the cost of a program is roughly how many instructions it retires - there is no pipeline to keep busy, no branch predictor to please, no cache hierarchy to schedule around. Add to that bounds checks for each memory access, gas metering for bounded execution, and it is substantially different from silicon implementation.

LLVM has no way to describe this today. Every RISC-V -mcpu/-mtune describes hardware, so projects in this space either take a tuning aimed at a machine they are not, or build one by hand out of flags meant for real cores. Both are happening right now:

  • PolkaVM’s rustc target spec sets +auipc-addi-fusion,+ld-add-fusion,+lui-addi-fusion,+xtheadcondmov on generic-rv64 - LLVM’s macro-fusion tuning features, plus a T-Head conditional-move extension for branchless selects. (riscv64emac-unknown-none-polkavm.json @ 28d4606)
    Also note that those fusion features have since been renamed to fusion-auipc-addi, fusion-ld-add and fusion-lui-addi, further complicating maintenance.
  • SP1 passes -C llvm-args=-misched-prera-direction=bottomup and -misched-postra-direction=bottomup to steer the machine scheduler directly. (crates/build/src/command/utils.rs @ 9c94078)
  • libriscv tells users to rebuild their toolchain without the C extension for a 20-25% interpreter speedup, and to unroll loops in the guest. (README @ e121610)

These are reasonable workarounds for something missing, and they are fragile: they lean on internal flags and on features whose meaning is tied to specific hardware.

Who runs RISC-V in software

Sandboxing and embedding. libriscv sandboxes game scripts and request handlers, with a pre-decoding threaded interpreter and optional binary translation. RVVM is a desktop-class emulator with a tracing JIT.

Blockchain VMs. PolkaVM (Polkadot) recompiles RISC-V at link time into its own bytecode, run by an interpreter or an AOT recompiler. CKB-VM (Nervos) runs RV64IMC+B in a pre-decoding assembly interpreter. Cartesi runs RV64GC well enough to boot Linux.

zkVMs. RISC Zero (RV32IM), SP1 (RV64IM), ZisK (RV64IMA), Jolt (RV64IMAC), OpenVM (RV32IM). The Ethereum Foundation’s zkvm-standards work proposes a shared guest target, riscv64im_zicclsm-unknown-none-elf, with the compressed extension explicitly excluded.

Two things recur: the cost unit is the dispatch, not the cycle - several of these meter gas per instruction from an explicit cost table, so for them, it is not even an approximation.
And several actively want adjacent instruction pairs, because their decoder folds them into one dispatch: CKB-VM’s macro-op fusion (RFC-0033/0049) covers auipc+jalr, lui+addiw and add/sltu carry chains. A scheduler tuned for hardware will happily split exactly those pairs apart.

What I have experimented with

One tuning and an interpreter-target subtarget feature, on LLVM main, exposed under the same three names upstream already uses for generic: generic-interpreter-rv32 and generic-interpreter-rv64 mirror generic-rv32/generic-rv64, and generic-interpreter mirrors generic as a tune-only entry, so -mtune=generic-interpreter pairs with any -march. None carries any extension of its own, so -march still decides the ISA. Two things differ from generic-rv64:

  • Branches are priced as unpredictable. A guest conditional branch becomes a host indirect jump whose target depends on guest data, and the host predictor cannot learn it. So getPredictableBranchThreshold() is 1 and jumps are marked expensive, and a compound condition gets computed instead of split into a chain of branches.
  • The macro-fusion tuning features are on - not because any hardware will fuse, but because the scheduling edges make fusible pairs more likely to come out adjacent.

That is all. No scheduling model of its own, inlining and unrolling untouched (at least not yet).

CoreMark

Same LLVM for both mid-level and codegen, same guest ISA (rv64imc_zba_zbb_zbs), 1000 iterations, run on a pre-decoding interpreter that fuses adjacent pairs.

instructions retired retired, interpreter fuses static insts fusable pairs score score, interpreter fuses
GCC 14.2 301,590,885 297,906,065 8387 30 2879 2898
LLVM generic-rv64 262,393,574 261,145,777 5077 79 3257 3215
LLVM generic-interpreter-rv64 250,983,705 232,314,852 5110 147 3413 3842

generic-interpreter-rv64 dispatches 4.35% fewer instructions than generic-rv64, and
11.5% fewer once the interpreter’s fusion pass runs on top, for 0.65% more code than
generic-rv64 and 39% less than GCC.

Scores are iterations/sec, higher is better. Best against best that is +17.9% over generic-rv64 and +32.6% over GCC.

Fusion is the other half. Running the interpreter’s fusion pass removes 7.44% of dispatches from the interpreter-targeted build but only 0.48% from the hardware-tuned one - below roughly 1% fused, its per-instruction overhead is not repaid at all. So the fusion tuning features are not a bonus on top of the instruction-count reduction, they are what makes running a fusion pass worth it in the first place.

Concretely, what my interpreter fuses is any adjacent pair matching one of its own rules, whether LLVM has a corresponding Fusion record not. Both builds, side by side, with the LLVM tune feature that puts each pair next to its partner:

pair generic-rv64 generic-interpreter-rv64 LLVM tune feature
auipc+addi 28 55 auipc-addi
sh[123]add(.uw)+load 15 27 shxadd-load
add+load/store 12 15 add-load; the store half has no feature
branch+c.mv 0 24 conditional-cmv-fusion
slli+srli(.uw) 9 10 bfext, shifted-zext
addi+addi 9 9 none
auipc+ld 3 3 auipc-load
and/andi+or/andi 3 4 logic-reg-reg, logic-reg-imm
total 79 147

Two things in there are the argument. Branch+c.mv goes from zero to 24 - the category does not exist in the hardware-tuned build - and it comes entirely from conditional-cmv-fusion, which is not a scheduling feature at all: it gates select lowering in RISCVISelLowering.cpp. And addi+addi fires nine times in both builds with no LLVM feature behind it, because none exists, the same is true of addi+sh[123]add and xor+rori(w), which do not appear in CoreMark but are the two most common adjacent pairs in the contract code I actually care about. Those are three SimpleFusion records that would cost nothing on hardware with the feature off.

Two real workloads

Interpreted, same setup, lower is better:

generic-rv64 generic-interpreter-rv64
BLAKE3, hash one chunk 16.6 µs 14.5 µs 12% faster
ed25519, verify a signature 687 µs 690 µs unchanged
Two things I tried that made it worse

Both of these seemed obviously right and were not. I worked through them with an LLM as a research assistant, which is also how most of the measurement plumbing got written.

A scheduling model that described an interpreter honestly. One serial dispatch slot, occupancy equal to the per-dispatch cost (a load pays for address arithmetic, a bounds check and a memory access, a multiply runs a longer handler), latency equal to that cost because a handler finishes before the next dispatch starts. It produced code about 10% slower than turning the machine scheduler off entirely. When occupancy equals latency there is no slack, so the scheduler has no gap to fill and no reason to keep a producer near its consumer - while still being free to stretch live ranges. On a 5×5-limb field multiply from popular curve25519-dalek Rust crate it left all fifty partial products live at once: 27 spill stores, where the generic hardware model interleaves each product with its consumer and spills nothing. A spill is a guest memory access, one of the most expensive dispatches there is.

Raising the inlining and unrolling thresholds, on the grounds that an interpreter has no instruction cache to protect. That made an ed25519 field inversion 64% slower at 3.7× the code: inlining a callee whose loop had high register demand made its trip count constant, the loop was then fully unrolled, and a body that already needed every register spilled hard.

Both came from the same assumption - that an interpreter’s cost is just its dispatch count - and both were caught only by timing, never by counting. If anyone can see how to express “reordering is free, register pressure is not” without lying about latency, I would like to hear it.

Prior art

CPU names in LLVM already describe things that are not silicon - WebAssembly’s lime1, x86-64-v2/v3/v4, AMDGPU’s gfxN-generic, RISC-V’s own generic-ooo - and RISCVUsage.md already anticipates more than one generic model: “Right now, we simply assign a scheduling model that is widely used by the community to generic. But in the future, we can create a standalone scheduling model for generic, or even create a generic model for each of the individual sectors. For example, a generic-embedded for embedded processors and a generic-server for server workloads.” (RISCVUsage.md, added by #167008, which also drew a report of a ~2.3% CoreMark regression on an embedded core.) This would be another of those sectors.

PolkaVM

PolkaVM was the obvious independent check: it hand-assembles a version of this tuning already, and it has both an interpreter and a recompiler. Only the cpu field of its target spec differs between the runs. Execution time, excluding compilation:

benchmark / backend PolkaVM’s tuning + generic-interpreter-rv64
pinky, 64-bit interpreter 72.05 ms 75.19 ms +4.4%
pinky, 64-bit recompiler 7127 µs 7191 µs +0.9%
prime-sieve, 64-bit interpreter 42.94 ms 42.93 ms −0.0%
prime-sieve, 64-bit recompiler 2918 µs 2885 µs −1.1%

Small, inconsistent in sign, and on single measurements that demonstrates no effect - except possibly pinky’s interpreter, which is worse.

Important note: PolkaVM does not execute RISC-V. Its linker lowers the RISC-V ELF into its own bytecode and runs its own optimization pipeline doing it - perform_inlining, perform_dead_code_elimination, perform_constant_propagation, perform_nop_elimination, perform_load_address_and_jump_fusion in program_from_elf.rs.
Whatever LLVM decided about selection and adjacency gets re-decided there.

So this is narrower than “software execution targets”. It is for consumers that dispatch RISC-V instructions directly. Where a consumer re-compiles RISC-V into something else first, its own middle-end decides and the RISC-V tuning washes out. PolkaVM is still evidence for the problem - it is hand-assembling hardware flags, and they may break with LLVM upgrades.

libriscv

So I ran a third one that does dispatch RISC-V directly: libriscv, which pre-decodes an ELF into a threaded dispatch table with no bytecode of its own in between. CoreMark again, but a hosted build of it against picolibc rather than the freestanding one above, 40000 iterations, nine pinned runs each on two machines, interpreter mode, -mcpu the only difference:

generic-rv64 generic-interpreter-rv64
guest instructions dispatched 10 492 964 537 10 036 546 491 −4.35%
host wall clock, median of 9, machine A 8455.8 ms 8223.2 ms −2.75%
host wall clock, median of 9, machine B 3577.5 ms 3485.9 ms −2.56%
.text 22 040 B 22 144 B +0.47%

Two things about this. The instruction count is a property of the codegen, not of the VM, and it lands on the same −4.35% as my own interpreter on a different CoreMark port. And roughly half of that converts into wall-clock time on someone else’s mature interpreter, from a change that is one -mcpu flag.

Across both machines, median, trimmed mean and fastest-run all fall between −1.8% and −2.8%.

Next steps

I’d like to collect thoughts about adding such software RISC-V target from someone more experienced. I’m not a compiler developer and while I try to make estimated guesses and LLMs can help me get quite far, it isn’t enough.

I’d be happy to open a PR with initial version of the target, but contributions from other RISC-V VMs and wider testing would be nice as well.

https://github.com/llvm/llvm-project/compare/main...nazar-pc:llvm-project:riscv-interpreter-target-forum-backup

LVM Weekly - #665, September 28th 2026
2

To be clear, per LLVM AI Tool Use Policy - LLVM, those LLMs were also behind a large part of this post?

3

Yes, part of it is written by me, and substantial chunk was originally authored by LLM before being reviewed and edited by myself.

I also used LLMs for research, experimentation and help with benchmarking scripts (especially for projects I’m not familiar with like libriscv). Most benchmarks were run on my machine (“machine A” is the only one done in the sandbox LLM was managing in the cloud) though and I checked to ensure they are what they appear to be.

This was most certainly “human in the loop” kind of situation and I’m responsible for the content presented.

I know it was a long post, but I wanted to include the benchmarking and experimentation results before bothering everyone with purely theoretical proposal.

4

I think people would be more likely to engage with such discussions if they weren’t prefaced with a wall of what reads like slop (in style, not necessarily (in)accuracy). At least speaking for myself, reading LLM-ese is quite draining and sucks any desire to engage materially compared with actual human interactions.

5

The post was planned and structured by me, not LLM, and it looks the way I intended it to look like regardless of who wrote which part of it originally. I apologize if it didn’t match expectations and will try to to do better next time.

6

It seems that you can just add a new subtarget feature and a new -mcpu/-mtune=interpreter (this is exactly what you did). Plus, you should compile your code with -Oz because it seems code size is all you need.

I am OK with adding such things, but please attend the next RISC-V LLVM bi-weekly sync-up meeting (RISC-V LLVM sync-up call Agenda / Notes - Google Docs) and discuss with other folks. :slight_smile:

1 Like
7

Size is certainly a factor, but -Oz is not actually faster to interpret from my testing despite being a lot smaller. Maybe for some workloads it is, but certainly not universally so.

I’ll try to join the next meeting, thanks!

8

-Oz optimizes for code size, not dynamic instruction count. This is clear in cases like inlining where Oz will likely inline a lot less resulting in more call overhead (and thus higher dynamic instruction count), but smaller code.

I’m not totally convinced that costing using purely executed instructions makes sense for an interpreter. I would expect some instructions to still be more expensive (e.g., vector instructions, especially if the interpreter isn’t using native SIMD to execute), but I guess it seems like a reasonable starting point.

9

Why? c.ld and ld are both a single instruction. And since interpreters don’t care about code size and/or ICache, why would it matter in the context of dynamic instruction count?

Is this just an issue related to libriscv’s own implementation?

I’d like to (re)throw the question of “what are the things that are good for hardware but generally harmful to interpreter?” here again: From your experiment I’m only convinced that splitting the branches might be bad for interpreter as it’ll increase the dynamic instruction counts while there is no branch predictors.

I don’t think which scheduling model you pick is relevant here. First, picking different scheduling models won’t affect dynamic instruction counts in most cases – it’s just a different order. The only cases that might affect dynamic IC is register spillings, but spillings still might happen anyway even if you don’t do any scheduling (i.e. NoSchedModel). That’s why I don’t think which scheduling model you pick matters a lot.

Regarding macro fushion (although it’s done by the scheduler, it’s separate from the scheduling model) I would like to point out that it only matters if your interpreters supports fusing instructions.

Overall, I have no objection on adding a new tuning CPU for interpreters. However, I wonder if we could use the new -mtune syntax – for example, -mtune=generic-rv64:+fusion-add-mem – to build your desired CPU instead of creating a new CPU name. Currently the specific example flag -mtune=generic-rv64:+fusion-add-mem is not valid because both the processor (generic-rv64 in this case) and tuning feature (i.e. fusion-add-mem) have to opt-in this new syntax (and neither of them do at this moment) but we can fix that.

A general suggestion, and also related to my previous comments, is that I think we should think critically on why certain features matters for interpreters. For example

  • PolkaVM enables fushion XYZ: okay but why? Is it because they also do fushion themselves? But what if an interpreter doesn’t do that, would it matter?
  • SP1 uses a different scheduler configuration…but why? If it’s a generic interpreter as you framed in the first paragraph then instruction order shouldn’t matter right?

Finally, I think dynamic IC is just one of the factors, because having more vector instructions can also give you lower dynamic IC, yet vectors might be pretty expensive to execute by the interpreter and degrade the overall performance.

1 Like
10

Can’t say about libriscv since I’m not familiar with its internals, but from the design of my interpreter there is a cost for supporting more than one instruction size. When all instructions are 32-bit, they can be stored in decoded form in a flat vector, fetching instructions is then a simple read at the offset. When some instructions are 16-bit, you need 2x the number of “slots” where instructions could be stored, doubling the amount of RAM needed for decoded instructions + some other overhead depending on the dispatch method (threading vs switch loop for example).

I didn’t claim it matters for dynamic instruction count, but it matters for overall interpreter performance. My design used more RAM but actually ran faster with compressed instructions, but I can imagine how it could be different for alternative architectures.

I tried NoSchedModel and observed a substantial performance regression. My understanding is that physical hardware has instruction-level parallelism and changing instruction order can better leverage various parts of the pipeline. In software implementation there is generally no such thing, so the ordering that benefits hardware is not necessarily the same one that benefits software and vice versa.

The best you can do is fusing instructions that, for example, touch the same registers (when output of one instruction is an input to the next). But even fusion operations that are efficient/useful in hardware and software are not necessarily the same.

Moreover, for software side there is a difference between interpreters, JIT and recompilers (PolkaVM for example has portable interpreter and AoT recompiler as you can see in the benchmarking results). Hence interpreter variant that I’m the most interested so far.

For the experiment I copied scheduling from generic, but I believe there should be opportunities to improve on it.

Also note that for the interpreter specifically each RISC-V guest instruction will result in ~10+ native host instructions, so while throughput could be considered to be 1, the latency is less obvious due to host-side instruction level parallelism and instructions with different inherent complexity may or may not take the same amount of time to execute in practice.

Enabling fusion benefits interpreters that do implement them since it allows to reduce effective dynamic instruction count. For interpreters that do not, it generally neither helps nor hurts performance from my testing. So PolkaVM developers must have implemented those particular fusions because they saw them as common enough in their workloads.

Very good question, but I’m not really familiar with internals of zero-knowledge VMs, I’ll try to reach out to developers and see if they can reply here.

Absolutely! There are too many unique instructions, their implementation is relatively large in terms of host code size and vectorization doesn’t seem to be of great quality as of LLVM 23.1 at least, which all contribute to vector instructions having limited positive impact and only on certain workloads.

For example, portable BLAKE3 implementation barely auto-vectorized at all (I’ll need to dig for numbers, but IIRC there was a handful of vector instructions overall in generated code), while ed25519 signature verification measurably improved performance with Zve64x extension (though vector instruction density was still relatively low).

11

-1, as I don’t think the premise that optimising dynamic instruction count → optimising interpreter performance is true.

As an example, QEMU TCC is very slow when interpreting RVV instructions, so optimising for dynamic instruction count doesn’t help. Similarly, there is no contract that libriscv will have better performance because it can use TCC as a JIT backend.

What would be more suitable is if there are specific interpreters with well-defined performance characteristics you want to optimise for. E.g. if we know we want to reduce gas metering in PolkaVM, then adding a specific PolkaVM subtarget that optimises for that would make sense.

12

I don’t think this is a fair comparison since the cost of vector instructions is incomparably higher than regular ones. As such, a huge decrease in dynamic instruction count would be necessary to compensate for it, and today’s auto-vectorization on RISC-V is not great either.

Yet, having this interpreter target like in my experiment should make output with vector instructions faster vs generic target all else being equal.

13

Curiously, Python does not do this; see https://godbolt.org/z/Efhh9rv14, or use the REPL:

Python 3.14.6 (main, Jul  5 2026, 10:35:55) [Clang 21.0.0 (https://android.googlesource.com/toolchain/llvm-project 5e96669f0 on android
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x, y):
...     if x > 1 and y < 0:
...         return "Foo"
...     else:
...         return "Bar"
...
>>> import dis
>>> dis.dis(f)
  1           RESUME                   0

  2           LOAD_FAST_BORROW         0 (x)
              LOAD_SMALL_INT           1
              COMPARE_OP             148 (bool(>))
              POP_JUMP_IF_FALSE       10 (to L1)
              NOT_TAKEN
              LOAD_FAST_BORROW         1 (y)
              LOAD_SMALL_INT           0
              COMPARE_OP              18 (bool(<))
              POP_JUMP_IF_FALSE        3 (to L1)
              NOT_TAKEN

  3           LOAD_CONST               1 ('Foo')
              RETURN_VALUE

  5   L1:     LOAD_CONST               2 ('Bar')
              RETURN_VALUE
>>>

Python 2.7.18 (default, Sep 12 2025, 12:48:39)
[GCC Android (13624864, +pgo, +bolt, +lto, +mlgo, based on r530567e) Clang 19.0 on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(x, y):
    if x > 1 and y < 0:
        return "Foo"
    else:
        return "Bar"

>>> import dis
>>> dis.dis(f)
  1           0 LOAD_FAST                0 (x)
              3 LOAD_CONST               1 (1)
              6 COMPARE_OP               4 (>)
              9 POP_JUMP_IF_FALSE       28
             12 LOAD_FAST                1 (y)
             15 LOAD_CONST               2 (0)
             18 COMPARE_OP               0 (<)
             21 POP_JUMP_IF_FALSE       28
             24 LOAD_CONST               3 ('Foo')
             27 RETURN_VALUE
        >>   28 LOAD_CONST               4 ('Bar')
             31 RETURN_VALUE
             32 LOAD_CONST               0 (None)
             35 RETURN_VALUE
>>>

(I would have tried earlier versions too, but I don’t have any handy. Looking at the history, I see that POP_JUMP_IF_FALSE was introduced in https://bugs.python.org/issue4715 (included in 3.1/2.7), so before that, they would have been JUMP_IF_FALSE instructions targetting one or more POP_TOP instructions, likely themselves followed by an unconditional jump of some sort …)

Does this mean Python has been doing it wrong?
You would think this would have been one of the first things dealt with in the “faster CPython” effort, if so …

What does “computed” even look like?

14

On sync-up call there wasn’t a consensus on adding a new target, so the proposal was to add subtarget feature for branch tuning. Turns out that particular feature already exists and I’ve since been looking at other changes from the diff that may not yet be reachable through CLI options.

I do not work with Python, but I think it is substantially different from C in that tiny snippet. < is not the same as < in C: the types are dynamic, the operator can be overloaded, have side effects or raise an exception, so the second comparison must not be evaluated unless the first one succeeded. From my research, COMPARE_OP is also much more expensive than POP_JUMP_IF_FALSE due to dynamic types. So what Python produces makes a lot of sense.

When both sides are side-effect-free and can’t fault, like integer comparisons in C, the transformation can be done safely. Something like this for if (a < b && ((c ^ d) + a) > 7):

Split                          Computed
  bge  a0, a1, .Lskip            xor  a2, a2, a3
  xor  a2, a2, a3                add  a2, a2, a0
  add  a0, a0, a2                slti a2, a2, 8
  li   a1, 8                     slt  a0, a0, a1
  blt  a0, a1, .Lskip            xori a1, a2, 1
                                 and  a0, a0, a1
                                 beqz a0, .Lskip

It is more instructions, but in the interpreter each of them costs about one dispatch, while every guest conditional branch makes the next dispatch depend on data. So it is the opposite trade-off from CPython. I might have overstated the prediction part in the original post, though. The benefit depends on how data-dependent the condition is and is implementation-specific, but I hope you get the idea.

I believe LLVM may already do this for cheap conditions even on regular targets.