Hi Peter,
Thanks so much for your help, again!
> > Hi,
> >
> > I've tried to compile Apache Httpd (and several other open source
> projects)
> > with clang using the dataflow sanitizer. I set the CC environment
> variable
> > to point to my clang installation and I specified the
> "-fsanitize=dataflow"
> > in the CFLAGS and/or LDFLAGS environment variables.
>
> FWIW, I normally set CC to "clang -fsanitize=foo" when I want to use a
> sanitizer.
>
> > However, the configure script fails when running its C compiler tests (in
> > Apache's case, the first problem occurs for the APR library). The
> following
> > is an excerpt of the config.log output:
> >
> > (...)
> > configure:5409: checking whether the C compiler works
> > configure:5431: /usr/local/llvm/build/Debug+Asserts/bin/clang
> > -fsanitize=dataflow conftest.c >&5
>
> This path name would appear to indicate that you are building LLVM/Clang
> with the makefiles. As I mentioned before, this doesn't work well at the
> moment with dfsan (sorry). Can you please try building LLVM/Clang with
> CMake?
>
>
Thank you, I just figured out that I was mistakenly using my old version of
compiler-rt (the one built with gnu configure/make). I think I got confused
with my own tests and I'm deeply sorry for this. However, although I'm now
able to compile nginx with dfsan, I still have a problem when compiling
Apache httpd (specifically Apache APR, a library used by Apache httpd).
Here is the error I got from running make:
clang -fsanitize=dataflow -lssl -lcrypto -lcrypt -lpthread \
-fsanitize=dataflow -o ab ab.lo
/home/frederico/dev/tests/llvm/httpd-2.4.6/srclib/apr-util/libaprutil-1.la/home/frederico/dev/tests/llvm/httpd-2.4.6/srclib/apr-util/xml/expat/
libexpat.la /home/frederico/dev/tests/llvm/httpd-2.4.6/srclib/apr/
libapr-1.la -lcrypt -lpthread
ab.o: In function `main':
ab.c:(.text+0xa1): undefined reference to `dfs$SSLv23_client_method'
ab.c:(.text+0x2cb7): undefined reference to `dfs$SSLv23_client_method'
ab.c:(.text+0x2da1): undefined reference to `dfs$SSLv3_client_method'
ab.c:(.text+0x2e8b): undefined reference to `dfs$TLSv1_1_client_method'
ab.c:(.text+0x2f75): undefined reference to `dfs$TLSv1_2_client_method'
ab.c:(.text+0x305f): undefined reference to `dfs$TLSv1_client_method'
ab.c:(.text+0x45f9): undefined reference to `dfs$CRYPTO_set_mem_functions'
ab.c:(.text+0x4604): undefined reference to `dfs$SSL_load_error_strings'
ab.c:(.text+0x4609): undefined reference to `dfs$SSL_library_init'
ab.c:(.text+0x46d5): undefined reference to `dfs$BIO_new_fp'
ab.c:(.text+0x479c): undefined reference to `dfs$BIO_new_fp'
ab.c:(.text+0x47fa): undefined reference to `dfs$SSL_CTX_new'
ab.c:(.text+0x48af): undefined reference to `dfs$BIO_printf'
ab.c:(.text+0x4921): undefined reference to `dfs$ERR_print_errors'
ab.c:(.text+0x49d5): undefined reference to `dfs$SSL_CTX_ctrl'
ab.c:(.text+0x4a7c): undefined reference to `dfs$SSL_CTX_ctrl'
ab.c:(.text+0x4bcd): undefined reference to `dfs$SSL_CTX_set_cipher_list'
ab.c:(.text+0x4d10): undefined reference to `dfs$ERR_print_errors_fp'
ab.c:(.text+0x4df7): undefined reference to `dfs$SSL_CTX_set_info_callback'
ab.o: In function `dfs$ssl_state_cb':
ab.c:(.text+0xb74c): undefined reference to `dfs$SSL_alert_type_string_long'
ab.c:(.text+0xb766): undefined reference to `dfs$SSL_alert_desc_string_long'
ab.c:(.text+0xb795): undefined reference to `dfs$BIO_printf'
ab.c:(.text+0xb842): undefined reference to `dfs$SSL_state'
ab.c:(.text+0xb878): undefined reference to `dfs$SSL_state_string_long'
ab.c:(.text+0xb8b0): undefined reference to `dfs$BIO_printf'
ab.c:(.text+0xb98d): undefined reference to `dfs$SSL_state_string_long'
ab.c:(.text+0xb9c7): undefined reference to `dfs$BIO_printf'
(...)
This suggests to me that I also have to compile OpenSSL using dfsan. So, I
tried to compile it with calng + dfsan, without success:
make[2]: Entering directory `/home/frederico/dev/tests/llvm/openssl/apps'
( :; LIBDEPS="${LIBDEPS:--L.. -lssl -L.. -lcrypto -ldl}";
LDCMD="${LDCMD:-clang -fsanitize=dataflow}";
LDFLAGS="${LDFLAGS:--DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN
-DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wall -DOPENSSL_IA32_SSE2
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM
-DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM}"; LIBPATH=`for x in $LIBDEPS; do
echo $x; done | sed -e 's/^ *-L//;t' -e d | uniq`; LIBPATH=`echo $LIBPATH |
sed -e 's/ /:/g'`; LD_LIBRARY_PATH=$LIBPATH:$LD_LIBRARY_PATH ${LDCMD}
${LDFLAGS} -o ${APPNAME:=openssl} openssl.o verify.o asn1pars.o req.o
dgst.o dh.o dhparam.o enc.o passwd.o gendh.o errstr.o ca.o pkcs7.o crl2p7.o
crl.o rsa.o rsautl.o dsa.o dsaparam.o ec.o ecparam.o x509.o genrsa.o
gendsa.o genpkey.o s_server.o s_client.o speed.o s_time.o apps.o s_cb.o
s_socket.o app_rand.o version.o sess_id.o ciphers.o nseq.o pkcs12.o pkcs8.o
pkey.o pkeyparam.o pkeyutl.o spkac.o smime.o cms.o rand.o engine.o ocsp.o
prime.o ts.o srp.o ${LIBDEPS} )
dgst.o: In function `dfs$dgst_main':
dgst.c:(.text+0x2ef5): undefined reference to `dfs$OPENSSL_cleanse'
enc.o: In function `dfs$enc_main':
enc.c:(.text+0x3539): undefined reference to `dfs$OPENSSL_cleanse'
ca.o: In function `dfs$ca_main':
ca.c:(.text+0x449d): undefined reference to `dfs$OPENSSL_cleanse'
s_server.o: In function `dfs$sv_body':
s_server.c:(.text+0x1033b): undefined reference to `dfs$OPENSSL_cleanse'
s_client.o: In function `dfs$s_client_main':
s_client.c:(.text+0xab2b): undefined reference to `dfs$OPENSSL_cleanse'
s_client.o:s_client.c:(.text+0xab59): more undefined references to
`dfs$OPENSSL_cleanse' follow
speed.o: In function `dfs$speed_main':
speed.c:(.text+0x7f9d): undefined reference to `dfs$RC4'
speed.c:(.text+0x8ec7): undefined reference to `dfs$AES_cbc_encrypt'
speed.c:(.text+0x93d7): undefined reference to `dfs$AES_cbc_encrypt'
speed.c:(.text+0x98e7): undefined reference to `dfs$AES_cbc_encrypt'
speed.c:(.text+0xaa8e): undefined reference to `dfs$AES_encrypt'
speed.c:(.text+0xb297): undefined reference to `dfs$Camellia_cbc_encrypt'
speed.c:(.text+0xb7a7): undefined reference to `dfs$Camellia_cbc_encrypt'
speed.c:(.text+0xbcb7): undefined reference to `dfs$Camellia_cbc_encrypt'
speed.c:(.text+0x14d76): undefined reference to `dfs$RC4_options'
apps.o: In function `dfs$password_callback':
apps.c:(.text+0x1af6): undefined reference to `dfs$OPENSSL_cleanse'
apps.c:(.text+0x1bbf): undefined reference to `dfs$OPENSSL_cleanse'
apps.c:(.text+0x1c84): undefined reference to `dfs$OPENSSL_cleanse'
version.o: In function `dfs$version_main':
version.c:(.text+0x8a6): undefined reference to `dfs$RC4_options'
../libssl.a(s2_lib.o): In function `dfs$ssl2_free':
s2_lib.c:(.text+0x75c): undefined reference to `dfs$OPENSSL_cleanse'
../libssl.a(s3_srvr.o): In function `dfs$ssl3_get_client_key_exchange':
s3_srvr.c:(.text+0x125be): undefined reference to `dfs$OPENSSL_cleanse'
s3_srvr.c:(.text+0x12b42): undefined reference to `dfs$OPENSSL_cleanse'
s3_srvr.c:(.text+0x12c98): undefined reference to `dfs$OPENSSL_cleanse'
s3_srvr.c:(.text+0x13c80): undefined reference to `dfs$OPENSSL_cleanse'
../libssl.a(s3_clnt.o):s3_clnt.c:(.text+0x12c6e): more undefined references
to `dfs$OPENSSL_cleanse' follow
../libcrypto.a(cryptlib.o): In function `dfs$OPENSSL_cpuid_setup':
cryptlib.c:(.text+0x206d): undefined reference to `dfs$OPENSSL_ia32_cpuid'
cryptlib.c:(.text+0x20d5): undefined reference to `dfs$OPENSSL_ia32_cpuid'
../libcrypto.a(mem.o): In function `dfs$CRYPTO_realloc_clean':
mem.c:(.text+0x1e5a): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(md5_dgst.o): In function `dfs$MD5_Update':
md5_dgst.c:(.text+0x2b6): undefined reference to
`dfs$md5_block_asm_data_order'
md5_dgst.c:(.text+0x358): undefined reference to
`dfs$md5_block_asm_data_order'
../libcrypto.a(md5_dgst.o): In function `dfs$MD5_Transform':
md5_dgst.c:(.text+0x676): undefined reference to
`dfs$md5_block_asm_data_order'
../libcrypto.a(md5_dgst.o): In function `dfs$MD5_Final':
md5_dgst.c:(.text+0x784): undefined reference to
`dfs$md5_block_asm_data_order'
md5_dgst.c:(.text+0x8a4): undefined reference to
`dfs$md5_block_asm_data_order'
../libcrypto.a(sha1dgst.o): In function `dfs$SHA1_Update':
sha1dgst.c:(.text+0x2b6): undefined reference to `dfs$sha1_block_data_order'
sha1dgst.c:(.text+0x358): undefined reference to `dfs$sha1_block_data_order'
../libcrypto.a(sha1dgst.o): In function `dfs$SHA1_Transform':
sha1dgst.c:(.text+0x676): undefined reference to `dfs$sha1_block_data_order'
../libcrypto.a(sha1dgst.o): In function `dfs$SHA1_Final':
sha1dgst.c:(.text+0x784): undefined reference to `dfs$sha1_block_data_order'
sha1dgst.c:(.text+0x8ab): undefined reference to `dfs$sha1_block_data_order'
../libcrypto.a(sha1_one.o): In function `dfs$SHA1':
sha1_one.c:(.text+0xcb): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(sha256.o): In function `dfs$SHA224':
sha256.c:(.text+0x423): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(sha256.o): In function `dfs$SHA256_Update':
sha256.c:(.text+0x716): undefined reference to `dfs$sha256_block_data_order'
sha256.c:(.text+0x7b8): undefined reference to `dfs$sha256_block_data_order'
../libcrypto.a(sha256.o): In function `dfs$SHA256_Final':
sha256.c:(.text+0xbc1): undefined reference to `dfs$sha256_block_data_order'
sha256.c:(.text+0xce3): undefined reference to `dfs$sha256_block_data_order'
../libcrypto.a(sha256.o): In function `dfs$SHA256':
sha256.c:(.text+0x1693): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(sha256.o): In function `dfs$SHA256_Transform':
sha256.c:(.text+0x1746): undefined reference to
`dfs$sha256_block_data_order'
../libcrypto.a(sha512.o): In function `dfs$SHA512_Final':
sha512.c:(.text+0x465): undefined reference to `dfs$sha512_block_data_order'
sha512.c:(.text+0x6bb): undefined reference to `dfs$sha512_block_data_order'
../libcrypto.a(sha512.o): In function `dfs$SHA512_Update':
sha512.c:(.text+0x12ac): undefined reference to
`dfs$sha512_block_data_order'
sha512.c:(.text+0x12f7): undefined reference to
`dfs$sha512_block_data_order'
../libcrypto.a(sha512.o): In function `dfs$SHA512_Transform':
sha512.c:(.text+0x1506): undefined reference to
`dfs$sha512_block_data_order'
../libcrypto.a(sha512.o): In function `dfs$SHA384':
sha512.c:(.text+0x1776): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(sha512.o): In function `dfs$SHA512':
sha512.c:(.text+0x1a16): undefined reference to `dfs$OPENSSL_cleanse'
../libcrypto.a(wp_dgst.o): In function `dfs$WHIRLPOOL_BitUpdate':
wp_dgst.c:(.text+0x5ac): undefined reference to `dfs$whirlpool_block'
wp_dgst.c:(.text+0x6c5): undefined reference to `dfs$whirlpool_block'
wp_dgst.c:(.text+0x7b7): undefined reference to `dfs$whirlpool_block'
wp_dgst.c:(.text+0xa66): undefined reference to `dfs$whirlpool_block'
wp_dgst.c:(.text+0xdd0): undefined reference to `dfs$whirlpool_block'
../libcrypto.a(wp_dgst.o):wp_dgst.c:(.text+0x11e0): more undefined
references to `dfs$whirlpool_block' follow
../libcrypto.a(aes_misc.o): In function `dfs$AES_set_encrypt_key':
aes_misc.c:(.text+0x3b): undefined reference to
`dfs$private_AES_set_encrypt_key'
../libcrypto.a(aes_misc.o): In function `dfs$AES_set_decrypt_key':
aes_misc.c:(.text+0x6b): undefined reference to
`dfs$private_AES_set_decrypt_key'
(...)
Does this mean that I also have to manually compile the libraries that
OpenSSL depends on?
I'm sure that there is a better way to do this, perhaps using the "Special
Case List" mentioned in the dfsan documentation. I was wondering if it's
possible to define the appropriate label propagation behavior of dfsan at
the OpenSSL interface level (more or less like you do for glibc, without
going deeper into the library internals). If this is possible, could you
please point me to the right direction to do this?
Please correct me if I made any incorrect conclusions.
I believe that the undefined symbols here belong to OpenSSL, and are defined
in .s files, so they are not subject to instrumentation. You will need to use
a special case list to tell the instrumentation pass about these functions.
Unfortunately we do not currently have a good way of managing special case
list files for dependent libraries. To start with you can probably just edit
the special case list file installed in your prefix or write a script that
does it for you.
The first step would be to declare each of these functions as uninstrumented,
e.g.
fun:OPENSSL_cleanse=uninstrumented
fun:RC4=uninstrumented
[etc.]
This will cause dfsan to emit a runtime warning when any of these functions
are called. You can then define the label propagation in the abilist file as
needed, as is currently done for glibc. (Note that you will need to arrange
for any custom functions to be compiled without instrumentation.)
> > unsigned long*, unsigned long, unsigned long)'
> >
> (...)(.data.rel.ro._ZTIN11__sanitizer17ThreadContextBaseE[_ZTIN11__sanitizer17ThreadContextBaseE]+0x0):
> > undefined reference to `vtable for __cxxabiv1::__class_type_info'
>
> This is the same problem that you had before, which I believe is fixed now.
> Are you sure that you have updated compiler-rt?
>
I had exported the wrong clang version to my PATH.
By the way, I'm exporting only the bin and libs folders to my PATH. Is that
all I need?
Yes, that should be enough. (I don't think the lib directory is needed though.)
Thanks,