Use of NewPM C API with pipeline parsing callbacks

IR & Optimizations
1

Hi all,

I’m the maintainer of LLVM.jl, the Julia wrapper for the LLVM C APIs. I’m working on improving our NewPM support, and trying to switch away from our API extensions (that expose most of the C++ API to C and thus Julia) to the official API, LLVMRunPasses. This is mostly working fine, but I’m running into an issue trying to run Julia’s custom passes and pipelines, wich are registered with LLVM by calling PassBuilder::registerPipelineParsingCallback. The problem here is that the notion of a PassBuilder isn’t exposed to the C API, and LLVMRunPasses just instantiates a temporary one.

What is the official/intended way to use the NewPM C API with custom passes and pipelines that require parsing callbacks? For running Julia passes, I’m currently vendoring a version of LLVMRunPasses that takes an additional callback argument, but that’s not ideal.

2

There hasn’t been a need for it yet, so we don’t have anything in the C API that does that. Although this seems a bit complicated for a C API, especially when all the pass manager boilerplate is C++.

How did you do this with the legacy pass manager via the C API?

Are you trying to test just a pass in isolation with the C API? Or add it to the optimization pipeline? Or both?

3

With the old pass manager, we added Julia’s passes to the pass manager by calling LLVMAdd* functions that we added to the Julia compiler library. We originally did that as well for the new pass manager, but we were told in C-API for NewPM that the string-based (i.e. PassBuilder-based) interface is preferable, is why I’m looking into making LLVMRunPasses work.

I don’t see this as something particularly complicated for the C API; I’m just looking for a way to interface with “out-of-tree” passes that have been implemented using the LLVM C++ APIs much like how you would otherwise invoke LLVM’s own passes through the C API (i.e. using strings). In other words, this doesn’t necessarily need anything extra in the C API. For example, if Julia would have a way to register these callbacks globally, instead of per-PassBuilder, I could probably use LLVMRunPasses as-is.

I’m building a pipeling that consists of regular LLVM passes, and custom passes that are part of the Julia compiler. Well, actually, I’m also adding LLVM passes that have been implemented in Julia (i.e. not written in C++) to the mix, but that’s outside of the scope of this question.

4

Registering callbacks globally is explicitly against the design of the new pass manager. What’s the issue with registering per-PassBuilder?

5

Sorry, I lost track of your reply.

The issue is that I’m using the C API, which doesn’t expose the PassBuilder objects. I’m working around this by vendoring a version of LLVMRunPasses where I do have access to the PassBuilder and can execute the callbacks appropriately: LLVM.jl/deps/LLVMExtra/lib/NewPM.cpp at ed30abdaf2bf26c1308585c8a89137b2a58479fa · maleadt/LLVM.jl · GitHub. But that’s not particularly clean, because it requires exposing the C++ callbacks from Julia’s compiler and passing it to the libLLVMExtra C API.