Load MachineFunctionPass plugin from library in llc?

Hi all,

I am working on creating a MachineFunctionPass to perform an analysis on X86 code. After a bit of trouble, I was able to get my pass compiling and running correctly in llc. However, since a machine pass is compiled directly into the code generator, rerunning “make” across the LLVM build tree involves re-linking most of the major libraries and executables. This is a rather intense process, both in terms of CPU usage and memory (some of the bigger binaries, especially clang, use as much as 5 GB peak memory during linking). With a powerful multicore machine and plenty of RAM, this is doable, but still very time-consuming, and seems wastefully repetitive.

For a front-end pass, I can load the pass from a dynamic shared library using “opt –l path_to_library”; thus I only need to rebuild the pass itself to test a code change. There does not appear to be such an option in llc for dynamically linking a machine pass as a plugin. Does anyone know of a way to achieve something similar?

At the very least, it would be nice to be able to just re-compile the parts of the LLVM tree I need (perhaps just llc). It seems particularly odd that clang, which is a front-end, would need to be relinked for a code change in a back-end pass. (clang, and some of its related components, are responsible for the lion’s share of the time-consuming CPU/memory-intensive linking phase.)

FYI, I am using the Autotools (./configure, make) process to build LLVM, but have also successfully built with CMake and would be willing to try suggestions involving either.

Any advice/suggestions would be greatly appreciated!

Thanks,

Ethan Johnson

Ethan J. Johnson

Computer Science PhD student, Systems group, University of Rochester

ejohns48@cs.rochester.edu

ethanjohnson@acm.org

PGP pubkey available from public directory or on request

Hi all,

I am working on creating a MachineFunctionPass to perform an analysis on X86 code. After a bit of trouble, I was able to get my pass compiling and running correctly in llc. However, since a machine pass is compiled directly into the code generator, rerunning “make” across the LLVM build tree involves re-linking most of the major libraries and executables. This is a rather intense process, both in terms of CPU usage and memory (some of the bigger binaries, especially clang, use as much as 5 GB peak memory during linking). With a powerful multicore machine and plenty of RAM, this is doable, but still very time-consuming, and seems wastefully repetitive.

For a front-end pass, I can load the pass from a dynamic shared library using “opt –l path_to_library”; thus I only need to rebuild the pass itself to test a code change. There does not appear to be such an option in llc for dynamically linking a machine pass as a plugin. Does anyone know of a way to achieve something similar?

There is unfortunately no way to do that currently.

At the very least, it would be nice to be able to just re-compile the parts of the LLVM tree I need (perhaps just llc). It seems particularly odd that clang, which is a front-end, would need to be relinked for a code change in a back-end pass. (clang, and some of its related components, are responsible for the lion’s share of the time-consuming CPU/memory-intensive linking phase.)

clang the project is the front end, but clang the executable program is the full compiler, including all of optimization, codegen, object emission, etc..

FYI, I am using the Autotools (./configure, make) process to build LLVM, but have also successfully built with CMake and would be willing to try suggestions involving either.

This, however, can be done, at least w/ CMake+ninja (probably other permutations, too, but that’s the one I use these days). Just do “ninja llc” and it will build just what’s required for llc. Same for any of the other tools.

Also enabling shared libraries "cmake -DBUILD_SHARED_LIBS=ON" reduces link times dramatically in my experience.

- Matthias

Hi all,

I am working on creating a MachineFunctionPass to perform an analysis on X86 code. After a bit of trouble, I was able to get my pass compiling and running correctly in llc. However, since a machine pass is compiled directly into the code generator, rerunning “make” across the LLVM build tree involves re-linking most of the major libraries and executables. This is a rather intense process, both in terms of CPU usage and memory (some of the bigger binaries, especially clang, use as much as 5 GB peak memory during linking). With a powerful multicore machine and plenty of RAM, this is doable, but still very time-consuming, and seems wastefully repetitive.

For a front-end pass, I can load the pass from a dynamic shared library using “opt –l path_to_library”; thus I only need to rebuild the pass itself to test a code change. There does not appear to be such an option in llc for dynamically linking a machine pass as a plugin. Does anyone know of a way to achieve something similar?

There is unfortunately no way to do that currently.

At the very least, it would be nice to be able to just re-compile the parts of the LLVM tree I need (perhaps just llc). It seems particularly odd that clang, which is a front-end, would need to be relinked for a code change in a back-end pass. (clang, and some of its related components, are responsible for the lion’s share of the time-consuming CPU/memory-intensive linking phase.)

clang the project is the front end, but clang the executable program is the full compiler, including all of optimization, codegen, object emission, etc…

FYI, I am using the Autotools (./configure, make) process to build LLVM, but have also successfully built with CMake and would be willing to try suggestions involving either.

This, however, can be done, at least w/ CMake+ninja (probably other permutations, too, but that’s the one I use these days). Just do “ninja llc” and it will build just what’s required for llc. Same for any of the other tools.

Also enabling shared libraries “cmake -DBUILD_SHARED_LIBS=ON” reduces link times dramatically in my experience.

  • Matthias

Thanks for the tips! I will definitely try building just llc, since that is all I need for testing/debugging the pass.

Out of curiosity, what exactly does BUILD_SHARED_LIBS do? Does it move most of the shared code to .so files and create binaries that load them at run time? If I go that route, is there anything special I need to do to make sure the binaries can “find” the shared libraries at run time? (I do not have root access on my development machine, so “make install” to put the shared libs in a standard location isn’t a really good option.)

Thanks,

Ethan Johnson

From: Matthias Braun [mailto:mbraun@apple.com]
Sent: Thursday, July 2, 2015 6:51 PM
To: Jim Grosbach
Cc: Ethan J. Johnson; llvmdev@cs.uiuc.edu
Subject: Re: [LLVMdev] Load MachineFunctionPass plugin from library in llc?

Hi all,

I am working on creating a MachineFunctionPass to perform an analysis on X86 code. After a bit of trouble, I was able to get my pass compiling and running correctly in llc. However, since a machine pass is compiled directly into the code generator, rerunning “make” across the LLVM build tree involves re-linking most of the major libraries and executables. This is a rather intense process, both in terms of CPU usage and memory (some of the bigger binaries, especially clang, use as much as 5 GB peak memory during linking). With a powerful multicore machine and plenty of RAM, this is doable, but still very time-consuming, and seems wastefully repetitive.

For a front-end pass, I can load the pass from a dynamic shared library using “opt –l path_to_library”; thus I only need to rebuild the pass itself to test a code change. There does not appear to be such an option in llc for dynamically linking a machine pass as a plugin. Does anyone know of a way to achieve something similar?

There is unfortunately no way to do that currently.

At the very least, it would be nice to be able to just re-compile the parts of the LLVM tree I need (perhaps just llc). It seems particularly odd that clang, which is a front-end, would need to be relinked for a code change in a back-end pass. (clang, and some of its related components, are responsible for the lion’s share of the time-consuming CPU/memory-intensive linking phase.)

clang the project is the front end, but clang the executable program is the full compiler, including all of optimization, codegen, object emission, etc..

FYI, I am using the Autotools (./configure, make) process to build LLVM, but have also successfully built with CMake and would be willing to try suggestions involving either.

This, however, can be done, at least w/ CMake+ninja (probably other permutations, too, but that’s the one I use these days). Just do “ninja llc” and it will build just what’s required for llc. Same for any of the other tools.

Also enabling shared libraries "cmake -DBUILD_SHARED_LIBS=ON" reduces link times dramatically in my experience.

- Matthias

Thanks for the tips! I will definitely try building just llc, since that is all I need for testing/debugging the pass.

Out of curiosity, what exactly does BUILD_SHARED_LIBS do? Does it move most of the shared code to .so files and create binaries that load them at run time? If I go that route, is there anything special I need to do to make sure the binaries can “find” the shared libraries at run time? (I do not have root access on my development machine, so “make install” to put the shared libs in a standard location isn’t a really good option.)

Yep, that’s exactly what it does. The load paths are relative to the main executable, so everything should “just work” without having to do any path munging or installation tricks.

-Jim

Good to know – thanks! It has indeed been working for me. Anecdotally, I can’t say it’s helped too much with build times – now it just takes a long time building “libX”, instead of X; but it does feel a little bit faster.

Maybe my observations are a bit misleading since building shared libraries consolidates a lot of code into a single linker product, rendering “make –jX” multithreading less effective. It spends a lot of time “bottlenecked” on linking big things like libclang: it launches a wave of linker tasks, and most of them finish quickly, leaving a single “big” task holding things up at the end. “make” has to wait for it to complete before it can launch a new wave of parallel build tasks (whether compiling, linking, etc.). I’m not sure if this is because of dependencies between components, or perhaps just because “make” isn’t very smart about multithreading. Maybe CMake would be smarter here? Or perhaps ninja?

I should be able to improve build times more by disabling the unit tests – I noticed those are responsible for a good portion of the time spent on linking (they don’t seem to use the shared libraries), and since I’m just writing an analysis pass which doesn’t actually modify anything, checking for regressions in the rest of LLVM isn’t so important (not at this stage anyway).

Thanks again,

Ethan Johnson