LIBOMP: TSAN functionality requested but not available

Jan_Studeny · May 16, 2019, 1:24pm

Hello everyone,
As the default clang is outputting false positives when running openmp program I wanted to compile openmp with TSAN (thread sanitizer) support. Unfortunately when I tried to compile openmp I got an error that says: "TSAN functionality requested but not available". What does it mean not available? Should I install something extra (what if it is the case)? (maybe helps: currently when compiling`clang -fsanitize=thread -g -O1 tiny_race.c` I get no error). Running on macOS 10.14.4 (18E226). Here is the output of `cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLIBOMP_TSAN_SUPPORT=1 ../openmp` (the openmp repository is cloned from https://github.com/llvm/llvm-project):

Thank you for any hints how to fix this error.
Regards,
Jan

-- The C compiler identification is Clang 8.0.0
-- The CXX compiler identification is Clang 8.0.0
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang
-- Check for working C compiler: /usr/local/opt/llvm/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++
-- Check for working CXX compiler: /usr/local/opt/llvm/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test OPENMP_HAVE_WERROR_FLAG
-- Performing Test OPENMP_HAVE_WERROR_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG - Success
-- Found PythonInterp: /usr/bin/python (found version "2.7.10")
-- Cannot find llvm-lit.
-- Please put llvm-lit in your PATH, set OPENMP_LLVM_LIT_EXECUTABLE to its full path, or point OPENMP_LLVM_TOOLS_DIR to its directory.
CMake Warning at cmake/OpenMPTesting.cmake:22 (message):
The check targets will not be available!
Call Stack (most recent call first):
cmake/OpenMPTesting.cmake:40 (find_standalone_test_dependencies)
CMakeLists.txt:49 (include)

-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG
-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG - Success
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG - Success
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG - Success
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG - Success
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG - Failed
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG - Success
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG - Success
-- Performing Test LIBOMP_HAVE_MMIC_FLAG
-- Performing Test LIBOMP_HAVE_MMIC_FLAG - Failed
-- Performing Test LIBOMP_HAVE_M32_FLAG
-- Performing Test LIBOMP_HAVE_M32_FLAG - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Found Perl: /usr/bin/perl (found version "5.18.2")
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS - Failed
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS - Success
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE - Success
-- Looking for include files windows.h, psapi.h
-- Looking for include files windows.h, psapi.h - not found
-- Looking for EnumProcessModules in psapi
-- Looking for EnumProcessModules in psapi - not found
CMake Error at runtime/cmake/LibompUtils.cmake:26 (message):
LIBOMP: TSAN functionality requested but not available
Call Stack (most recent call first):
runtime/CMakeLists.txt:325 (libomp_error_say)

-- Configuring incomplete, errors occurred!
See also "/tmp/llvm-project/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/llvm-project/build/CMakeFiles/CMakeError.log".

Joachim_Protze · May 16, 2019, 1:38pm

Hello Jan,

the logic to detect whether the TSan annotations might be supported on your platform is quite simple in the OpenMP runtime. We only tested this on x86_64 Linux, so this is the only platform that is flagged supported in

runtime/cmake/config-ix.cmake:280

You might just remove/modify that lines. But I remember that there are some issues with the weak implementation workflow in the TSan annotations.

I would suggest you to try this branch instead:

I will push this to upstream soon, but did not find the time to update my patch in phabricator yet.

Best
Joachim

Jan_Studeny · May 16, 2019, 2:55pm

Hi, thanks for quick reply. I have tried lifting the restriction (setting LIBOMP_HAVE_TSAN_SUPPORT to TRUE) in both branches but unfortunately I got 1 or 2 errors when compiling from`llvm-project` or from `PRUNERS/openmp`. For the completeness they are at the bottom of the message. My Clang version is 8.0.0. Is there some plan to support the TSAN on macOS soon? Otherwise I will try to manually compile archer and hope it will work (already tried to install through spack but it failed - was hanging over an hour on installation of perl).

Thanks for help.

Jan

**pruners/openmp**
CMake Warning (dev):
Policy CMP0042 is not set: MACOSX_RPATH is enabled by default. Run "cmake
--help-policy CMP0042" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

MACOSX_RPATH is not specified for the following targets:

omptarget

[ 26%] Building CXX object runtime/src/CMakeFiles/omp.dir/kmp_environment.cpp.o
/Users/janstudeny/metagenome_lossless_dbg/metagraph/experiments/lossless_dbg/playground/openmp/libomptarget/src/omptarget.cpp:75:36: error: field of type 'HostDataToTargetListTy::iterator'
(aka '__list_iterator') has private constructor
LookupResult() : Flags({0,0,0}), Entry(0) {}
^
/usr/local/opt/llvm/bin/../include/c++/v1/list:303:14: note: implicitly declared private here
explicit __list_iterator(__link_pointer __p) _NOEXCEPT : __ptr_(__p) {}

[ 92%] Building CXX object runtime/src/CMakeFiles/omp.dir/tsan_annotations.cpp.o
In file included from /Users/janstudeny/metagenome_lossless_dbg/metagraph/experiments/lossless_dbg/playground/openmp/runtime/src/kmp_ftn_cdecl.cpp:33:
/Users/janstudeny/metagenome_lossless_dbg/metagraph/experiments/lossless_dbg/playground/openmp/runtime/src/kmp_ftn_entry.h:939:9: error: definition of builtin function 'omp_is_initial_device'
xexpand(FTN_IS_INITIAL_DEVICE)( void )
^
/Users/janstudeny/metagenome_lossless_dbg/metagraph/experiments/lossless_dbg/playground/openmp/runtime/src/kmp_ftn_os.h:110:50: note: expanded from macro 'FTN_IS_INITIAL_DEVICE'
#define FTN_IS_INITIAL_DEVICE omp_is_initial_device

**llvm-project**

[ 50%] Building CXX object runtime/src/CMakeFiles/omp.dir/kmp_threadprivate.cpp.o
/tmp/llvm-project/openmp/runtime/src/kmp_runtime.cpp:8031:52: warning: 'syscall' is deprecated: first deprecated in macOS 10.12 - syscall(2) is unsupported; please switch to a supported interface. For
      SYS_kdebug_trace use kdebug_signpost(). [-Wdeprecated-declarations]
    rc = __kmp_str_buf_print(field_buffer, format, __kmp_gettid());
                                                   ^
/tmp/llvm-project/openmp/runtime/src/kmp_wrapper_getpid.h:25:24: note: expanded from macro '__kmp_gettid'
#define __kmp_gettid() syscall(SYS_thread_selfid)
                       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/unistd.h:745:6: note: 'syscall' has been explicitly marked deprecated here
int      syscall(int, ...);

[ 88%] Building CXX object runtime/src/CMakeFiles/omp.dir/kmp_version.cpp.o
In file included from /tmp/llvm-project/openmp/runtime/src/kmp_ftn_cdecl.cpp:31:
/tmp/llvm-project/openmp/runtime/src/kmp_ftn_entry.h:971:33: error: definition of builtin function 'omp_is_initial_device'
int FTN_STDCALL KMP_EXPAND_NAME(FTN_IS_INITIAL_DEVICE)(void) {
                                ^
/tmp/llvm-project/openmp/runtime/src/kmp_ftn_os.h:105:31: note: expanded from macro 'FTN_IS_INITIAL_DEVICE'
#define FTN_IS_INITIAL_DEVICE omp_is_initial_device

Joachim_Protze · May 16, 2019, 3:00pm

Hi Jan,

sorry, I missed to add the information that with the archer_80 branch, you should NOT set LIBOMP_TSAN_SUPPORT=on.
This branch comes with a tool, that is loaded at runtime and should also provide better results, than the annotations activated by LIBOMP_TSAN_SUPPORT=on.

I also suggest to

export TSAN_OPTIONS="ignore_noninstrumented_modules=1"

to avoid some false positive reports in the OpenMP runtime.

- Joachim