How to port symcc to clang/llvm-13?

Alberto · December 16, 2021, 10:38pm

Hi all,
Few days ago I have discovered the symcc[1] project. This project, via an llvm pass, is able to modify the IR code and to inject calls to a backend which allows symbolic execution. I can use it with clang/llvm 11 but not with the version 13.

I was wondering if maybe the new pass manager uses llvm passes in a different way … so I have created a small pass which injects a call to printf in each function and I’m able to use it via opt. Now my question is: is it possible to run the same pass via clang and just obtain the modified IR code? I’d like to avoid to use opt if not mandatory. Is it possible to do it or the new pass manager forces me to use opt?

How would you fix this situation in symcc?

If someone could tell me how to load a ModulePass in clang-13 would be great.

Thanks a lot
Alberto

[1] https://github.com/eurecom-s3/symcc

Min-Yih_Hsu · December 17, 2021, 1:11am

It’s a lot easier to load custom pass plugins into clang with the new PassManager actually:

clang -fpass-plugin=<path to plugin> ...

Note that needs to be an absolute path.

-Min

Alberto · December 17, 2021, 6:56am

Hi Min-Yih,
thanks for your email. I have searched online on how to build clang plugins and I found [1]. I wanted to use the example that prints all the function names but, despite the fact that I followed the instructions, I could not compile it

I searched again and I found [2] and hopefully In this case I could build and run the basic hello-world example. I then tried to just replace the HelloWorld.cpp with the example that print the functions names but no luck. I compile but I have an error message. The following paragraph shows the entire test:

➜ build git:(main) ✗ make
Scanning dependencies of target HelloWorld
[ 50%] Building CXX object CMakeFiles/HelloWorld.dir/HelloWorld.cpp.o
[100%] Linking CXX shared library libHelloWorld.so
[100%] Built target HelloWorld
➜ build git:(main) ✗ clang-13 -cc1 -load ./libHelloWorld.so -plugin hello-world $CLANG_TUTOR_DIR/test/HelloWorld-basic.cpp
➜ build git:(main) ✗ $Clang_DIR/bin/clang -cc1 -load ./libHelloWorld.so -plugin hello-world $CLANG_TUTOR_DIR/test/HelloWorld-basic.cpp
(clang-tutor) file: /home/alberto/Desktop/projects/llvm/clang-tutor//test/HelloWorld-basic.cpp
(clang-tutor) count: 3
➜ build git:(main) ✗ cd …
➜ HelloWorld git:(main) ✗ wget https://raw.githubusercontent.com/llvm/llvm-project/main/clang/examples/PrintFunctionNames/PrintFunctionNames.cpp
–2021-12-17 06:45:54-- https://raw.githubusercontent.com/llvm/llvm-project/main/clang/examples/PrintFunctionNames/PrintFunctionNames.cpp
Resolving raw.githubusercontent.com (raw.githubusercontent.com)… 185.199.110.133, 185.199.109.133, 185.199.108.133, …
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 4504 (4.4K) [text/plain]
Saving to: ‘PrintFunctionNames.cpp’

PrintFunctionNames.cp 100%[========================>] 4.40K --.-KB/s in 0s

2021-12-17 06:45:54 (11.3 MB/s) - ‘PrintFunctionNames.cpp’ saved [4504/4504]

➜ HelloWorld git:(main) ✗ mv HelloWorld.cpp HelloWorld.cpp.backup
➜ HelloWorld git:(main) ✗ mv PrintFunctionNames.cpp HelloWorld.cpp
➜ HelloWorld git:(main) ✗ rm -rf build
➜ HelloWorld git:(main) ✗ take build
➜ build git:(main) ✗ cmake …/
– The C compiler identification is GNU 10.2.1
– The CXX compiler identification is GNU 10.2.1
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Check for working C compiler: /usr/bin/cc - skipped
– Detecting C compile features
– Detecting C compile features - done
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Check for working CXX compiler: /usr/bin/c++ - skipped
– Detecting CXX compile features
– Detecting CXX compile features - done
– Performing Test Terminfo_LINKABLE
– Performing Test Terminfo_LINKABLE - Success
– Found Terminfo: /usr/lib/x86_64-linux-gnu/libtinfo.so
– Found ZLIB: /usr/local/lib/libz.so (found version “1.2.11”)
– Configuring done
– Generating done
– Build files have been written to: /home/alberto/Desktop/progetti/llvm/clang-tutor/HelloWorld/build
➜ build git:(main) ✗ make
Scanning dependencies of target HelloWorld
[ 50%] Building CXX object CMakeFiles/HelloWorld.dir/HelloWorld.cpp.o
[100%] Linking CXX shared library libHelloWorld.so
[100%] Built target HelloWorld
➜ build git:(main) ✗ $Clang_DIR/bin/clang -cc1 -load ./libHelloWorld.so -plugin hello-world $CLANG_TUTOR_DIR/test/HelloWorld-basic.cpp
error: unable to find plugin ‘hello-world’
➜ build git:(main) ✗

Any idea on how to solve it? I think a little github repo with an example on how to do it as standalone project would be beneficial to others as well.

Thanks a lot for all
Alberto

[1] https://clang.llvm.org/docs/ClangPlugins.html

[2] https://github.com/banach-space/clang-tutor

Min-Yih_Hsu · December 18, 2021, 12:52am

Both links you provided are for Clang plugin not LLVM plugin. Examples of clang plugins including modifying AST or registering custom #pragma directives.

To my best understanding your original question was asking about LLVM plugin, which works on LLVM IR.

Best,
-Min

Min-Yih_Hsu · December 18, 2021, 1:06am

Hi Min:

I am interested in your discussion, so I had it a try. Seem the option -fpass-plugin=<path to plugin> to clang brings the plugin to Clang Plugins(frontend).

A Clang plugin, which works on AST, custom pragma or attribute, is different from a LLVM pass plugin (works on LLVM IR). This flag is designated for LLVM pass plugin.

There is a comment around the keyword, -fpass-plugin, “Forward -fpass-plugin=name.so to -cc1”. Did I understand it correctly?

The code you referenced is part of the clang driver, who simply relays -fpass-plugin flag to the frontend. The flag is eventually consumed by clang CodeGen at this palce: https://github.com/llvm/llvm-project/blob/713ee230f884651afd17ab7b910e85a992ce406b/clang/lib/CodeGen/BackendUtil.cpp#L1321
Which uses llvm::PassPlugin::Load to load your custom pass plugin into the LLVM pass pipeline.

Best,
-Min

Alberto · December 18, 2021, 8:36am

Hi,
I think I got confused. Just clarify: my main goal is to be able to port symcc[1] to llvm-13.

Since I’m confused I went back to the beginning I have experimented with and HelloWorld [2] LLVM pass. As you can see in the following paragraph all went well:

root@eaa014e3667a:~/llvm-tutor/build# opt-13 -load-pass-plugin ./libHelloWorld.so -passes=hello-world -disable-output output.ll
(llvm-tutor) Hello from: foo
(llvm-tutor) number of arguments: 1
(llvm-tutor) Hello from: bar
(llvm-tutor) number of arguments: 2
(llvm-tutor) Hello from: fez
(llvm-tutor) number of arguments: 3
(llvm-tutor) Hello from: main
(llvm-tutor) number of arguments: 2
root@eaa014e3667a:~/llvm-tutor/build#

At this point I wanted to load directly the plugin via clang using the -fplugin parameter but I wasn’t able to do it. I tried the following command:

root@eaa014e3667a:~/llvm-tutor/build# clang -fpass-plugin=/root/llvm-tutor/build/libHelloWorld.so …/inputs/input_for_hello.c -o hello
root@eaa014e3667a:~/llvm-tutor/build# file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=ba237b79f2b2bcd362e894657b1e203af914aa9c, for GNU/Linux 3.2.0, not stripped
root@eaa014e3667a:~/llvm-tutor/build#

I was expecting the same output as before where the number of parameters were printed.

With some verbosity the interesting part was

“/usr/lib/llvm-13/bin/clang” -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name input_for_hello.c -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/root/llvm-tutor/build -resource-dir /usr/lib/llvm-13/lib/clang/13.0.1 -internal-isystem /usr/lib/llvm-13/lib/clang/13.0.1/include -internal-isystem /usr/local/include -internal-isystem /usr/bin/…/lib/gcc/x86_64-linux-gnu/10/…/…/…/…/x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir=/root/llvm-tutor/build -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -fpass-plugin=/root/llvm-tutor/build/libHelloWorld.so -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/input_for_hello-c52362.o -x c …/inputs/input_for_hello.c

symcc loads the plugin in this way:

➜ build git:(master) ✗ tail -n 8 symcc

exec $compiler
-Xclang -load -Xclang “$pass”
“$@”
-L"$runtime_dir"
-lSymRuntime
-Wl,-rpath,"$runtime_dir"
-Qunused-arguments
➜ build git:(master) ✗

How am I supposed to load the plugin via clang so I can just have the same output? I want to avoid the use of opt.

Thanks a lot for your help
Alberto

[1] https://github.com/eurecom-s3/symcc

[2] https://github.com/banach-space/llvm-tutor

Min-Yih_Hsu · December 18, 2021, 9:03am

Hi,
I think I got confused. Just clarify: my main goal is to be able to port symcc[1] to llvm-13.

Since I’m confused I went back to the beginning I have experimented with and HelloWorld [2] LLVM pass. As you can see in the following paragraph all went well:

root@eaa014e3667a:~/llvm-tutor/build# opt-13 -load-pass-plugin ./libHelloWorld.so -passes=hello-world -disable-output output.ll
(llvm-tutor) Hello from: foo
(llvm-tutor) number of arguments: 1
(llvm-tutor) Hello from: bar
(llvm-tutor) number of arguments: 2
(llvm-tutor) Hello from: fez
(llvm-tutor) number of arguments: 3
(llvm-tutor) Hello from: main
(llvm-tutor) number of arguments: 2
root@eaa014e3667a:~/llvm-tutor/build#

At this point I wanted to load directly the plugin via clang using the -fplugin parameter

Correct, that is expected, -fplugin is for Clang plugin not LLVM pass plugin.

but I wasn’t able to do it. I tried the following command:

root@eaa014e3667a:~/llvm-tutor/build# clang -fpass-plugin=/root/llvm-tutor/build/libHelloWorld.so …/inputs/input_for_hello.c -o hello
root@eaa014e3667a:~/llvm-tutor/build# file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=ba237b79f2b2bcd362e894657b1e203af914aa9c, for GNU/Linux 3.2.0, not stripped
root@eaa014e3667a:~/llvm-tutor/build#

I was expecting the same output as before where the number of parameters were printed.

The HelloWorld example you mentioned is using llvm::PassBuilder::registerPipelineParsingCallback. Passes registered in this way are not visible to clang.
More specifically, if you registered a Pass via registerPipelineParsingCallback, you need to call PassBuilder::parsePassPipeline with a textual pass pipeline description. But clang doesn’t use that function at all. The opt tool is using it (via the —passes flag) though.

Try to use other PassPipline registration functions like PassPipeline::registerPipelineStartEPCallback or PassPipeline::registerOptimizerLastEPCallback.

Also, when the optimization level is set to -O0 (which is the default one), every IR function is annotated with optnone, which prevents a function from being visited by any LLVM Pass. So be sure to add -Xclang -disable-O0-optnone flag on clang to turn this off (or using optimization levels other than -O0).

Best,
-Min

dougpuob · December 18, 2021, 4:25pm

Hi Min:

I am interested in the discussion. I modified the code from clang-tutor to register with PassPipeline::registerPipelineStartEPCallback. The plugin can be loaded with clang with -fpass-plugin=<path to plugin> option. Thank you for your suggestions and help.

Alberto · December 18, 2021, 5:14pm

Hi Douglas,
Just be clear, did you modify the helloworld example from llvm-tutor not clang-tutor right?

Would you mind to share the entire code, the clang full command and output? It’d help me to solve a lot of time.

Imo it could be added to the website as a tutorial.

Thanks
Alberto

dougpuob · December 19, 2021, 2:37am

Hi Alberto:

You are right, llvm-tutor. I am happy to share my change with you. I also put the patch file with complete change in this mail.

llvm::PassPluginLibraryInfo getHelloWorldPluginInfo() {
return {LLVM_PLUGIN_API_VERSION, "HelloWorld", LLVM_VERSION_STRING,
[](PassBuilder &PB) {
errs() << "registerPipelineStartEPCallback" << "\n";
PB.registerPipelineStartEPCallback(
[](ModulePassManager &MPM, OptimizationLevel Level) {
FunctionPassManager FPM;
FPM.addPass(HelloWorld());
MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM)));
});
}};
}

❯ $LLVM_DIR/bin/clang --version
clang version 14.0.0 ([https://github.com/llvm/llvm-project.git](https://github.com/llvm/llvm-project.git) eb052f6b8f787847827f9cc9ff8e09f954cb0a44)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/dougpuob/petzone/llvm/installed/main/debug//bin

❯ make
Scanning dependencies of target HelloWorld
[ 50%] Building CXX object CMakeFiles/HelloWorld.dir/HelloWorld.cpp.o
[100%] Linking CXX shared library libHelloWorld.so
[100%] Built target HelloWorld

❯ cat /home/dougpuob/petzone/llvm/testdata/20211218--symcc/input.c
#include <stdio.h>

int main() {
printf("Hello Input !!! \n");
return 0;
}⏎

❯ $LLVM_DIR/bin/clang \
-fpass-plugin=/home/dougpuob/petzone/llvm/testdata/20211218--symcc/llvm-tutor.git/build/libHelloWorld.so \
/home/dougpuob/petzone/llvm/testdata/20211218--symcc/input.c
CodeGenOpts.PassPlugins
PassPlugin::Load()
registerPipelineStartEPCallback
registerPipelineStartEPCallback()
(llvm-tutor) Hello from: main
(llvm-tutor) number of arguments: 0
(llvm-tutor) Opcode : alloca
(llvm-tutor) Opcode : store
(llvm-tutor) Opcode : call
(llvm-tutor) Opcode : ret

0001-Register-plugin-with-registerPipelineStartEPCallback.patch (2.16 KB)

Alberto · December 19, 2021, 7:17am

Hello,

Thanks a lot, I’ll try it now!

Alberto