C-API for NewPM

[vchuravy]

Is anyone aware of efforts to expose the new pass-manager interface through the C-API?
we will need an interface like that for Julia and so I don’t want to duplicate efforts

x-ref: C-API for NewPM · Issue #344 · maleadt/LLVM.jl · GitHub

[nikic]

The C API for the new pass manager is defined in llvm-project/PassBuilder.h at main · llvm/llvm-project · GitHub.

[vchuravy]

We have found the PassBuilder not flexible enough, and in Julia as an example we mostly bypass it.

It’s in particular a little bit strange (and cumbersome) to use strings to program the API.

[aeubanks]

The problem with a simple API for adding passes is that the new PM requires explicit nesting of different pass types, e.g. you need to explicitly create an adaptor that transforms function passes to a module pass. The legacy PM did that, which people generally agree was error prone.

We could create APIs that explicitly handle the nesting. That’s kinda annoying but perhaps doable.

I assume when you say “not flexible enough” you mean dynamically turning on/off some passes is hard to do with strings?

[vchuravy]

Yeah the Julia pipeline itself has grown organically over the years and certain things we don’t need (IPO)

We also use the C-API in the Julia ecosystem for Enzyme and specializing the pipeline for GPU compilation

[nikic]

I don’t think we should expose individual passes (and all the infra needed to work with them, like pass managers and pass adaptors) via the C API. This would expose a very large API surface.

If you use a custom optimization pipeline and the textual representation is not good enough, then you should implement your custom pipeline in C++ and only expose the construction of the whole pipeline via the C API. It looks like you already do that.

[rnk]

I’m not sure how to feel about this choice. I think the principal behind the C API is that we want to expose a relatively stable API that lets users build a standard, AoT compiler on top of LLVM. In practice, most major frontends end up tweaking the pass pipeline (correct me if wrong). Clang, Swift, Rust, and Julia all do this. They all use the C++ API to adjust the pipeline.

If our goal is to provide frontends with stability, I’m not sure we can do that with the stringly-typed pass pipeline DSL. We can’t keep it stable because we add and remove passes all the time. If we recommend frontends use this API, how will they recover from errors when pass name lookup fails?

Google is interested in finding a way to stabilize the API surface that Rust uses from LLVM, since we are trying to get all of our LLVM based toolchains to use the same version of LLVM for cross-language LTO, sanitizers, and various other ABI unstable features. I’ve been advocating for a plan of “figure out whatever C++ APIs Rust needs, put a high-level C wrapper around it, and contribute it to LLVM”. If the pipeline is just too tightly coupled with LLVM and we can’t provide any kind of stable API for it, maybe we have to give up, compromise, and accept this level of API instability.

I don’t want to push hard for a stable C API, since that’s a lot of work, but higher-level C++ APIs that reduce duplication between all of the major LLVM frontends would be a great place to start.

[vchuravy]

@nikic

If you use a custom optimization pipeline and the textual representation is not good enough, then you should implement your custom pipeline in C++ and only expose the construction of the whole pipeline via the C API. It looks like you already do that.

We do a lot of experimentation and customization of the pipeline. Not having to use C++ is a big boon for users of LLVM.jl (the Julia based LLVM interface).

If our goal is to provide frontends with stability, I’m not sure we can do that with the stringly-typed pass pipeline DSL.

Yeah, I feel the same. String based API’s make compatibility across API versions harder.

I don’t want to push hard for a stable C API, since that’s a lot of work.

I think in particularly for the PassManager the effort wouldn’t be to big (and the current situation is a regression from the old PM). From the Julia side we might make the effort to add this.

I should probably also do a pass over LLVM.jl/deps/LLVMExtra at master · maleadt/LLVM.jl · GitHub and figure out what pieces of that we should upstream. We use this shim-library to extend the API surface the C-API covers and to backport C-API components.

[nikic]

@rnk

I’m not sure how to feel about this choice. I think the principal behind the C API is that we want to expose a relatively stable API that lets users build a standard, AoT compiler on top of LLVM. In practice, most major frontends end up tweaking the pass pipeline (correct me if wrong). Clang, Swift, Rust, and Julia all do this. They all use the C++ API to adjust the pipeline.

Rust uses the standard optimization pipeline. The setup for how to do that with the NewPM has seen a lot of churn, in part because inserting sanitizer passes in all the right places is considered a frontend responsibility. If we could move sanitizer setup into PassBuilder (which is imho the right place to do it), that would allow sharing this logic across frontends and reduce the amount of churn.

@rnk

If our goal is to provide frontends with stability, I’m not sure we can do that with the stringly-typed pass pipeline DSL. We can’t keep it stable because we add and remove passes all the time. If we recommend frontends use this API, how will they recover from errors when pass name lookup fails?

I’m not really sure how that differs from a C API? If a pass is removed, that’s going to break explicit uses of that pass no matter how they look like.

The main way the C API for the NewPM is intended to be used is with the default pipelines, i.e. using something like "default<O3>" as the pipeline. That kind of usage should be stable.

@rnk

Google is interested in finding a way to stabilize the API surface that Rust uses from LLVM, since we are trying to get all of our LLVM based toolchains to use the same version of LLVM for cross-language LTO, sanitizers, and various other ABI unstable features. I’ve been advocating for a plan of “figure out whatever C++ APIs Rust needs, put a high-level C wrapper around it, and contribute it to LLVM”. If the pipeline is just too tightly coupled with LLVM and we can’t provide any kind of stable API for it, maybe we have to give up, compromise, and accept this level of API instability.

There are quite a few simple holes in the C API that don’t require any tricky design work, but just upstreaming lots of missing bits. Most things defined in rust/RustWrapper.cpp at master · rust-lang/rust · GitHub are like that.

[vchuravy]

Maybe to clarify I differentiate between PassBuilder which indeed as an C-API already,
and users (like Julia) who use the (Loop|Function|Module)PassMangager directly and bypass the PassBuilder for the most part.

For us PassBuilder is not flexible enough, it has a lot of extra capabilities that we don’t quite care about (like setting up IPO/LTO/…) and inserting passes is limited to the extension points. We have a pass for example that is doing allocation optimization and we would like to schedule passes like IRCE.

Since we live down-stream it easier for us to maintain/adjust/tune our own pipeline instead of relying on PassBuilder or changing it away from it’s clang/opt roots. I suspect this is also true for other front-ends that use LLVM (since otherwise I would assume Azul to have added an option to run IRCE, but maybe extension points is good enough for them).

Now the assumption thatPassBuilder is the only way of creating a pipeline has been frustrating, since backends use it to schedule their own passes using extension points. This is what motivated ⚙ D148561 Expose PassBuilder extension point callbacks so that we could schedule target passes using the PassBuilder EPs…

So most of the Julia based ecosystem uses the old pass-manager interface to schedule passes in a bespoke way, we even added infrastructure to write pure Julia passes and schedule them. All in a programmatic fashion.

So I want to provide functionality parity for the NewPM, and yes I could programmatically build up a large string…, then pass that to the PassBuilder and call that a day, but if I thought strings were a good programming interface I would use Perl We deal with API deprecations and removals anyhow so I am not to worried about churn caused by changes to the passes.