pie and sanitizers

We recently set up a new powerpc system with Ubuntu 17.10 for llvm development work. It works ok for llvm except that most of the msan and tsan tests fail. From a bit of experimentation and reading it looks like pie is the culprit.

Ubuntu now uses pie by default and they even have a note about it:

"Failures using ThreadSanitizer

Another source of build failures occurs when building with ThreadSanitizer (aka TSAN), which doesn't work with pie binaries. These packages need to disable via -no-pie. "

(from https://wiki.ubuntu.com/SecurityTeam/PIE)

So I tried forcing -no-pie on the msan amd tsan tests and the msan tests all worked but one (which seems to be failing for some other reason) and many of the failing tsan ones work as well (I need to further investigate the other failures).

Looking at the virtual addresses that are used with pie on this new system I don't think that I can adjust the ranges that msan and tsan use to accommodate things. For example on an older Ubuntu 16.04 system (with pie forced on) a trivial little test program run multiple times sees low memory addresses in the range

0x0000000100000984 to 0x000000013FFF0984

while on the 17.10 system

0x00000001002409B4 to 0x00001000FFF409B4

which is orders of magnitude larger.

I am not sure why the range is so large. On another non-Ubuntu system with similar kernel and binutils versions I don't see this. So, any ideas on how to handle this?

For immediate needs is there some way to locally add options when testing besides modifying the lit.cfg files?

We recently set up a new powerpc system with Ubuntu 17.10 for llvm
development work. It works ok for llvm except that most of the msan and
tsan tests fail. From a bit of experimentation and reading it looks like
pie is the culprit.

Ubuntu now uses pie by default and they even have a note about it:

"Failures using ThreadSanitizer

Another source of build failures occurs when building with ThreadSanitizer
(aka TSAN), which doesn't work with pie binaries. These packages need to
disable via -no-pie. "

(from https://wiki.ubuntu.com/SecurityTeam/PIE)

So I tried forcing -no-pie on the msan amd tsan tests and the msan tests all
worked but one (which seems to be failing for some other reason) and many of
the failing tsan ones work as well (I need to further investigate the other
failures).

Looking at the virtual addresses that are used with pie on this new system I
don't think that I can adjust the ranges that msan and tsan use to
accommodate things. For example on an older Ubuntu 16.04 system (with pie
forced on) a trivial little test program run multiple times sees low memory
addresses in the range

0x0000000100000984 to 0x000000013FFF0984

while on the 17.10 system

0x00000001002409B4 to 0x00001000FFF409B4

which is orders of magnitude larger.

I am not sure why the range is so large. On another non-Ubuntu system with
similar kernel and binutils versions I don't see this. So, any ideas on how
to handle this?

I'm pretty sure Ubuntu has cherry-picked a kernel change that
increases executable ASLR range. The same happened on x86_64 recently.
On x86 it only affected ASan, not MSan or TSan.

You two options are
(1) Fix the mapping somehow. 0x00001000FFF409B4 is still only ~1/8th
of the 48-bit address space, and theoretic upper limit is 1/3 (1x for
shadow and 1x for origins).
(2) Implement dynamic shadow. We've recently come up with a more
efficient way of doing it in ASan using ifunc resolver to put the
shadow start address directly in the GOT table, where it can be
accessed with just one PC-relative load. See PremapShadow(). That's
still a performance hit, of course.