Injecting customizations for TargetMachine pass

Hello llvm-dev,

I would like to be able to customise the X86TargetMachine but I couldn't find any proper way to do this from code external to the LLVM codebase. For the IR passes there are RegisterPass and RegisterStandardPasses. Have I overlooked an existing mechanism for extending TargetMachine passes or is this not really supported?

As background, I'm working on runtime inlining of calls made via function pointers, including C++ virtual functions. This uses a normal compiler like clang with some additional decoration of the IR before running llc to get the ahead-of-time compiled machine code. The decorated code can then re-compile parts of the IR at runtime by calling into LLJIT.

I use some low-level mechanisms to pass information between decorated functions alongside the normal platform ABI, and this is where I need to manipulate the target machine behaviour. The end result is to store some data in the instruction stream preceding the function return address, and pass a context pointer in r14.

Currently I have a workaround by adding a whole new target via RegisterTargetMachine. The implementation does the additional processing in runOnMachineFunction and delegates as much as possible to the real X86 target obtained via TargetRegistry::lookupTarget. This is especially messy because many of the TargetPassConfig virtual functions that I delegate to X86PassConfig are protected (e.g. addPreRegAlloc) so it's clearly not a "proper" way of doing what I want. The code in question is in https://github.com/drti/drti/blob/a204564d74ac5f8ad6946b1c96e10c84a1ffb97e/passes/drti-target.cpp and a general description of the runtime inliner is at https://github.com/drti/drti

Can anyone suggest a better way to do the TargetMachine customization? If necessary I could contribute some LLVM code changes to add extension points but would appreciate some guidance on which way to approach it first.

Thanks,
Raoul.

Yes, that’s exactly it - I have a small amount of extra code that I want to run on MachineFunctions within the X86 target machine implementation. I’ve tried a couple of ways around this and the easiest way I’ve come up with is very similar to the PassManagerBuilder::addGlobalExtension used for IR passes. I’ll post some code below showing the interface, in case anyone has any feedback before I write some tests for a patch submission.

My apologies for taking so long to reply, by the way.

The basic idea is to add an enum for different extension points within TargetPassConfig and keep a global list of per-target extensions. The interface changes look like the below. The existing addPass function has to be made public so the client code can use it from the callback function that it registers.

class TargetPassConfig : public ImmutablePass {

+public:

  • enum CodegenExtensionPointTy {
  • CGEP_EarlyAsPossible,
  • CGEP_BeforePreRegAlloc,
  • CGEP_AfterPreRegAlloc,
  • CGEP_BeforePostRegAlloc,
  • CGEP_AfterPostRegAlloc,
  • CGEP_BeforePreSched2,
  • CGEP_AfterPreSched2,
  • CGEP_BeforePreEmitPass,
  • CGEP_AfterPreEmitPass,
  • CGEP_BeforePreEmitPass2,
  • CGEP_AfterPreEmitPass2,
  • CGEP_LateAsPossible
  • };
  • using ExtensionFn = std::function<void(TargetPassConfig &)>;

private:
PassManagerBase *PM = nullptr;
AnalysisID StartBefore = nullptr;
@@ -343,6 +363,19 @@ public:
/// Returns the CSEConfig object to use for the current optimization level.
virtual std::unique_ptr getCSEConfig() const;

  • /// Add a global extension to be applied for the given Target TODO:
  • /// add ID and removal (cf commit
  • /// ab2300bc154f7bed43f85f74fd3fe31be71d90e0)
  • static void addGlobalExtension(const Target* Target,
  • CodegenExtensionPointTy CGEP, ExtensionFn Fn);
  • /// Add a pass to the PassManager if that pass is supposed to be run, as
  • /// determined by the StartAfter and StopAfter options. Takes ownership of the
  • /// pass.
  • /// @p verifyAfter if true and adding a machine function pass add an extra
  • /// machine verification pass afterwards.
  • void addPass(Pass *P, bool verifyAfter = true);

protected:

Then the client code can do something like this:

const Target* target = TargetRegistry::lookupTarget(
“x86_64-unknown-linux-gnu”, error);

TargetPassConfig::addGlobalExtension(
target, TargetPassConfig::CGEP_BeforePreRegAlloc,
&drti::X86DrtiTreenodePass::addSelf);

If anyone has any suggestions, please let me know

Thanks,
Raoul.