Using TSAN along with custom llvm IR pass

Project Infrastructure LLVM Dev List Archives
1

+llvm-dev mailing list

2

Hello,

I have written a custom pass to do some tasks and I want to run this pass on
a source code file along with TSAN instrumentation.

Steps I followed:
1. I compiled the code file with -fsanitize=thread and -emit-llvm to get the
byte code of the file
2. Used opt to run my custom pass on the TSAN instrumented .bc file
3. Run the final .bc file using lli with proper inputs

This gives me a "LLVM ERROR: Program used external function '__tsan_init'
which could not be resolved!"

Hi,

I don't know what is lli, but from the error message it seems that the
problem is that you don't link in tsan runtime (tsan runtime provides
__tsan_init function).
I think you need to compile you instrumented file into object file
first (.o), and then link it with clang -fsanitize=thread (which will
link in runtime).

3

Hello,

I tried that as well but there are no tsan warnings thrown at run time, even though there are data race error!

The steps I followed:

  1. Generate bitcode using ‘clang -emit-llvm’ command
  2. Using the ‘opt -load …’ command, I instrumented the bitcode file with my custom pass
  3. Using the command ‘opt -tsan …’, instrumented the output bitcode file from step 2
  4. Using the llvm backend compiler, ‘llc’, compiled the program to native assembly
  5. The output from step 4 is compiled again with ‘clang -fsanitize=thread’ (linking tsan here) for the final object file.

Executing this final output file works fine, except that it doesn’t throws any TSAN warnings!

In between, lli command is used to invoke the LLVM Interpreter.

4

Hi Nischai,

I see 3 possibilities:
1. Either the code is somehow ends up being non-instrumented, or
2. tsan does not consider what it sees a data race, or
3. the data race is masked by unrelated synchronization.

For 1 you can check that the resulting code in fact contains __tsan_* callbacks.
For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2
define, then it will print everything it sees (memory accesses and
synchronization). This output will allow us to figure out why it does
not report a race.

5

I think the code is not getting instrumented with tsan at all. In the ‘step 3’, when I instrument it using ‘opt -tsan …’, it gives a warning “WARNING: You’re attempting to print out a bitcode file.” I guess this method doesn’t instrument the code.

Is there any other way I can instrument the code with tsan?

6

I think the code is not getting instrumented with tsan at all. In the 'step
3', when I instrument it using 'opt -tsan ...', it gives a warning "WARNING:
You're attempting to print out a bitcode file." I guess this method doesn't
instrument the code.

Is there any other way I can instrument the code with tsan?

One option is to add your pass to llvm and execute clang with -fsanitize=thread.

7

Oh ok.
But is there any dependency when I change to this method, other than the way the pass is registered? Because when I used clang along with -Xclang to run the pass on the program, it is crashing!

8

I don't know.

9

Hi Dmitry,

I managed to get the program instrumented with tsan using ‘opt -tsan …’, but still it is not printing tsan warnings when I run the binary. When I instrument the same program with tsan during compilation using ‘clang -fsanitize=thread …’, without instrumenting my IR pass, it does give out tsan warnings!

10

Here is the script we use on build bots:

You need this option with -DCOMPILER_RT_DEBUG=ON
-DCOMPILER_RT_TSAN_DEBUG_OUTPUT=ON:

echo @@@BUILD_STEP build tsan with stats and debug output@@@
build_tsan "${TSAN_FULL_DEBUG_BUILD_DIR}" "-DCOMPILER_RT_DEBUG=ON
-DCOMPILER_RT_TSAN_DEBUG_OUTPUT=ON -DLLVM_INCLUDE_TESTS=OFF" gcc g++

11

OK. So if I am not wrong, I need to do a cmake on the llvm build directory, with the mentioned flags and then do make - which will basically build the whole llvm repo.