Hi,
I'm working on bringing up complete coverage for a Gentoo x32 "desktop"
system. I've been cooking up quite a few patches for various packages
to push upstream, but right now, the biggest blocker is the lack of
support for building with/codegen targeting x32 in llvm/clang. Since
the x32 patches were sent last year, I see support code has landed in
LLVM, and basic handling of 32-bit pointers in Clang, but no
elf32_x86_64 codegen or support for the -mx32 switch. I've been trying
to get x32 ABI support working in Clang and compiler-rt, I based off the
previous clang patch, bringing it up to date with the changes in trunk,
and hacked together handling of x32 "ARCH" support for compiler-rt.
(there must be a better way??)
Unfortunately, although the build succeeds as far as beginning to build
compiler-rt the resulting clang doesn't seem to respect the fact that
the target is x86_64-pc-linux-gnux32, nor that the "-mx32" opt has been
supplied. Something is clearly missing.
Can somebody please take a look at the patches, and look to see what I'm
missing/where I've gone wrong, or otherwise point me to a working
patchset/repo if I'm duplicating effort.
Patches to follow...
Clang patch for X32 support. Applies against current trunk.
--- ./tools/clang/include/clang/Driver/Options.td.orig 2013-05-16
21:51:51.286129820 +0000
+++ ./tools/clang/include/clang/Driver/Options.td 2013-05-16
21:53:24.875004239 +0000
@@ -841,6 +841,7 @@
HelpText<"Enable hexagon-qdsp6 backward compatibility">;
def m3dnowa : Flag<["-"], "m3dnowa">, Group<m_x86_Features_Group>;
def m3dnow : Flag<["-"], "m3dnow">, Group<m_x86_Features_Group>;
+def mx32 : Flag<["-"], "mx32">, Group<m_Group>, Flags<[DriverOption]>;
def m64 : Flag<["-"], "m64">, Group<m_Group>, Flags<[DriverOption]>;
def mabi_EQ : Joined<["-"], "mabi=">, Group<m_Group>;
def march_EQ : Joined<["-"], "march=">, Group<m_Group>;
--- ./tools/clang/lib/Driver/Driver.cpp.orig
+++ ./tools/clang/lib/Driver/Driver.cpp
@@ -1700,6 +1700,9 @@ static llvm::Triple computeTargetTriple(StringRef
DefaultTargetTriple,
if (Target.getArch() == llvm::Triple::ppc)
Target.setArch(llvm::Triple::ppc64);
}
+ } else if (Args.getLastArg(options::OPT_mx32)) {
+ if (Target.getArch() == llvm::Triple::x86_64)
+ Target.setEnvironment(llvm::Triple::GNUX32);
}
return Target;
--- ./tools/clang/lib/Basic/Targets.cpp.orig 2013-08-18
12:00:04.501402572 +0000
+++ ./tools/clang/lib/Basic/Targets.cpp 2013-08-18 12:08:11.086175660
+0000
@@ -2384,6 +2384,14 @@
Builder.defineMacro("__amd64");
Builder.defineMacro("__x86_64");
Builder.defineMacro("__x86_64__");
+ if (PointerWidth == 64 && getLongWidth() == 64) {
+ Builder.defineMacro("_LP64");
+ Builder.defineMacro("__LP64__");
+ } else if (PointerWidth == 32 && getLongWidth() == 32 &&
+ getIntWidth() == 32) {
+ Builder.defineMacro("_ILP32");
+ Builder.defineMacro("__ILP32__");
+ }
} else {
DefineStd(Builder, "i386", Opts);
}
@@ -3013,20 +3021,31 @@
class X86_64TargetInfo : public X86TargetInfo {
public:
X86_64TargetInfo(const llvm::Triple &Triple) : X86TargetInfo(Triple)
{
- LongWidth = LongAlign = PointerWidth = PointerAlign = 64;
+ const bool IsX32 = (getTriple().getEnvironment() ==
llvm::Triple::GNUX32);
+ LongWidth = LongAlign = PointerWidth = PointerAlign = IsX32 ? 32 :
64;
LongDoubleWidth = 128;
LongDoubleAlign = 128;
LargeArrayMinWidth = 128;
LargeArrayAlign = 128;
SuitableAlign = 128;
- IntMaxType = SignedLong;
- UIntMaxType = UnsignedLong;
- Int64Type = SignedLong;
+ if (IsX32) {
+ SizeType = UnsignedInt;
+ PtrDiffType = SignedInt;
+ IntPtrType = SignedInt;
+ } else {
+ IntMaxType = SignedLong;
+ UIntMaxType = UnsignedLong;
+ Int64Type = SignedLong;
+ }
RegParmMax = 6;
- DescriptionString =
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-"
X32 support patch for compiler-rt. Applies against current trunk.
--- projects/compiler-rt/make/platform/clang_linux.mk~ 2013-08-21
06:27:38.000000000 +0000
+++ projects/compiler-rt/make/platform/clang_linux.mk 2013-08-21
11:16:55.891621025 +0000
@@ -41,7 +41,18 @@
SupportedArches += x86_64
endif
else
- SupportedArches := x86_64
+ # x86-64 arch has two ABIs 64 bit x86-64 and 32 bit x32
+ ifeq ($(lastword $(subst -gnu, ,$(CompilerTargetTriple))),x32)
+ SupportedArches := x32
+ ifeq ($(call TryCompile,$(CC),$(test_source),-m64),0)
+ SupportedArches += x86_64
+ endif
+ else
+ SupportedArches := x86_64
+ ifeq ($(call TryCompile,$(CC),$(test_source),-mx32),0)
+ SupportedArches += x32
+ endif
+ endif
ifeq ($(call TryCompile,$(CC),$(test_source),-m32),0)
SupportedArches += i386
endif
@@ -74,6 +85,22 @@
Arch.lsan-x86_64 := x86_64
endif
+# Build runtime libraries for x32.
+ifeq ($(call contains,$(SupportedArches),x32),true)
+Configs += full-x32 profile-x32 san-x32 asan-x32 tsan-x32 \
+ msan-x32 ubsan-x32 ubsan_cxx-x32 dfsan-x32 lsan-x32
+Arch.full-x32 := x32
+Arch.profile-x32 := x32
+Arch.san-x32 := x32
+Arch.asan-x32 := x32
+Arch.tsan-x32 := x32
+Arch.msan-x32 := x32
+Arch.ubsan-x32 := x32
+Arch.ubsan_cxx-x32 := x32
+Arch.dfsan-x32 := x32
+Arch.lsan-x32 := x32
+endif
Not a real review, but it seems like this change needs tests -- at
least driver tests, and tests for predefined macros.
Dmitri
If you are just trying to get basic things working, you don't need to
port compiler-rt immediately. It is still pretty important (for
example, to support features like Address Sanitizer), but not critical
-- you will still get a useable Clang without building compiler-rt.
Dmitri
Having Clang working would of course be my primary goal, that said,
being able to compile compiler-rt is a pretty good test all is working
as it should be! 
It's also built/installed as part of the Gentoo
llvm ebuild when the clang useflag is enabled, so from that point of
view, I need to get it working.
Hi Steven,
This looks interesting and raises a number of questions
-
Does applying this patch actually bring working sanitizers to x32 platform?
That is, after you build the clang, does “clang -fsanitize=whatever foo.c” compile/link/run with expected results?
I doubt that, as there is some platform-specific code in all the sanitizers, and many of them heavily depend
on the address space layout. Porting TSan and MSan to 32-bit address space is especially hard, and we don’t plan
to do this anytime soon. I think it makes sense to build only the libraries that are expected to work on a given arch.
-
If we’re stepping on the path of porting some sanitizers to x32, it would be great to
setup a buildbot, make our tests pass cleanly, and make this bot public to catch regressions.
Do you have plans for that? Running sanitizer test suites also leads us to…
-
Do you plan to add support for building sanitizers on x32 to CMake build system? It would
make sense (and, in fact, I would start from there), as our testsuite can be run only in CMake builds.
Ah, I’ve replied in a different thread already. What Dmitri says - If you’re interesting in only building the Clang on x32 host, you may avoid checking out compiler-rt repo for now.
> > I've been trying
> > to get x32 ABI support working in Clang and compiler-rt, I based off
the
> > previous clang patch, bringing it up to date with the changes in trunk,
> > and hacked together handling of x32 "ARCH" support for compiler-rt.
> > (there must be a better way??)
>
> If you are just trying to get basic things working, you don't need to
> port compiler-rt immediately. It is still pretty important (for
> example, to support features like Address Sanitizer), but not critical
> -- you will still get a useable Clang without building compiler-rt.
>
> Dmitri
>
Having Clang working would of course be my primary goal, that said,
being able to compile compiler-rt is a pretty good test all is working
as it should be! It's also built/installed as part of the Gentoo
llvm ebuild when the clang useflag is enabled, so from that point of
view, I need to get it working.
I see, but it's not really good to build the libraries that are not
expected to work...I think you
should instead detect x32 host in the makefiles and avoid buidling
compiler-rt libraries in that case.
Note that compiler-rt contains arch/type-size dependent code, so it
might need porting in addition to build system changes.
Dmitri
Hi Steven,
This looks interesting and raises a number of questions
1) Does applying this patch actually bring working sanitizers to x32
platform?
That is, after you build the clang, does "clang -fsanitize=whatever foo.c"
compile/link/run with expected results?
I doubt that, as there is some platform-specific code in all the
sanitizers, and many of them heavily depend
on the address space layout. Porting TSan and MSan to 32-bit address space
is especially hard, and we don't plan
to do this anytime soon. I think it makes sense to build only the libraries
that are expected to work on a given arch.
I should have made clear this is very much a WIP. I expect to have make
an effort to port compiler-rt, but first I need to be able to generate
elf32_x86_64 objects with Clang. I confess, my method of porting code
to x32 so far has consisted of trying to build existing code and
modifying what breaks, or disabling features where inapplicable. I
haven't studied the code well enough to understand and predict where I'm
going to have make modification in advance. It's worked so far with
most things I've attempted but LLVM/Clang/compiler-rt is an order of
magnitude more complex than anything else I've poked at. The biggest
success I've had so far with this method is porting of Mozilla
Spidermonkey/js17.
2) If we're stepping on the path of porting some sanitizers to x32, it
would be great to
setup a buildbot, make our tests pass cleanly, and make this bot public to
catch regressions.
Do you have plans for that? Running sanitizer test suites also leads us
to...
I'm new to llvm/clang, I'm going to have to take guidance from others.
3) Do you plan to add support for building sanitizers on x32 to CMake build
system? It would
make sense (and, in fact, I would start from there), as our testsuite can
be run only in CMake builds.
I can do so, (locally switch to cmake) I have a little experience with
cmake so I should be able to figure it out. From my distro point of
view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
ebuild, but it hasn't happened yet. Perhaps this could be expedited, I
can contact the package maintainer.
I figured this would be the case, but decided to post my build system
changes as is. My plan was/is to look into making the compiler-rt code
work after failing to build it! Generally type-size errors get
detected with at least a warning, although that's admittedly pretty lazy
porting on my part, there was just no way I could read through that much
code in the last few days without trying to get something working
first! 
Certainly, if I can get to the point of a working Clang, but I find the
libraries are beyond my current ability to port, it might well be worth
making available what does work. But right now this is only building on
my local chroot, with the intention of then pushing it to my
experimental x32 portage overlay once I have something worth testing.
> Hi Steven,
>
> This looks interesting and raises a number of questions
>
> 1) Does applying this patch actually bring working sanitizers to x32
> platform?
> That is, after you build the clang, does "clang -fsanitize=whatever
foo.c"
> compile/link/run with expected results?
> I doubt that, as there is some platform-specific code in all the
> sanitizers, and many of them heavily depend
> on the address space layout. Porting TSan and MSan to 32-bit address
space
> is especially hard, and we don't plan
> to do this anytime soon. I think it makes sense to build only the
libraries
> that are expected to work on a given arch.
>
I should have made clear this is very much a WIP. I expect to have make
an effort to port compiler-rt, but first I need to be able to generate
elf32_x86_64 objects with Clang. I confess, my method of porting code
to x32 so far has consisted of trying to build existing code and
modifying what breaks, or disabling features where inapplicable. I
haven't studied the code well enough to understand and predict where I'm
going to have make modification in advance. It's worked so far with
most things I've attempted but LLVM/Clang/compiler-rt is an order of
magnitude more complex than anything else I've poked at. The biggest
success I've had so far with this method is porting of Mozilla
Spidermonkey/js17.
So, IIUC, if you're working on making LLVM/Clang/compiler-rt source tree
build on x32 host, I suggest your first step should be: disable building
compiler-rt
on x32 host One of the reasons: in makefile-based build compiler-rt
sources
are compiled with just-built Clang (I honestly don't know if Clang can
target x32
at the moment). Another reason: compiler-rt code is really arch-specific,
and even
if it compiles, it wouldn't work.
> 2) If we're stepping on the path of porting some sanitizers to x32, it
> would be great to
> setup a buildbot, make our tests pass cleanly, and make this bot public
to
> catch regressions.
> Do you have plans for that? Running sanitizer test suites also leads us
> to...
>
I'm new to llvm/clang, I'm going to have to take guidance from others.
> 3) Do you plan to add support for building sanitizers on x32 to CMake
build
> system? It would
> make sense (and, in fact, I would start from there), as our testsuite can
> be run only in CMake builds.
>
I can do so, (locally switch to cmake) I have a little experience with
cmake so I should be able to figure it out. From my distro point of
view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
ebuild, but it hasn't happened yet. Perhaps this could be expedited, I
can contact the package maintainer.
FTR, CMake would probably see that we our host is neither x86_64 nor i386,
and won't build anything in compiler-rt. If it wouldn't, we should go and
fix this.
> > Hi Steven,
> >
> > This looks interesting and raises a number of questions
> >
> > 1) Does applying this patch actually bring working sanitizers to x32
> > platform?
> > That is, after you build the clang, does "clang -fsanitize=whatever
> foo.c"
> > compile/link/run with expected results?
> > I doubt that, as there is some platform-specific code in all the
> > sanitizers, and many of them heavily depend
> > on the address space layout. Porting TSan and MSan to 32-bit address
> space
> > is especially hard, and we don't plan
> > to do this anytime soon. I think it makes sense to build only the
> libraries
> > that are expected to work on a given arch.
> >
> I should have made clear this is very much a WIP. I expect to have make
> an effort to port compiler-rt, but first I need to be able to generate
> elf32_x86_64 objects with Clang. I confess, my method of porting code
> to x32 so far has consisted of trying to build existing code and
> modifying what breaks, or disabling features where inapplicable. I
> haven't studied the code well enough to understand and predict where I'm
> going to have make modification in advance. It's worked so far with
> most things I've attempted but LLVM/Clang/compiler-rt is an order of
> magnitude more complex than anything else I've poked at. The biggest
> success I've had so far with this method is porting of Mozilla
> Spidermonkey/js17.
>
So, IIUC, if you're working on making LLVM/Clang/compiler-rt source tree
build on x32 host, I suggest your first step should be: disable building
compiler-rt
on x32 host One of the reasons: in makefile-based build compiler-rt
sources
are compiled with just-built Clang (I honestly don't know if Clang can
target x32
at the moment). Another reason: compiler-rt code is really arch-specific,
and even
if it compiles, it wouldn't work.
See accompanied Clang patch in other email. I've attempted to get Clang
to target x32, I'm also suspicious that x32 built clang isn't correctly
working to produce 64-bit code, but that's a somewhat different bug that
I haven't yet attempted to address. Without modifying Clang to target
x32, the x32 *built* Clang fails to compile compiler-rt with -m64, I
need help decoding the crash backtrace to figure out where that bug
lies, but it's not related to (or helped by) my patch.
>
> > 2) If we're stepping on the path of porting some sanitizers to x32, it
> > would be great to
> > setup a buildbot, make our tests pass cleanly, and make this bot public
> to
> > catch regressions.
> > Do you have plans for that? Running sanitizer test suites also leads us
> > to...
> >
> I'm new to llvm/clang, I'm going to have to take guidance from others.
>
> > 3) Do you plan to add support for building sanitizers on x32 to CMake
> build
> > system? It would
> > make sense (and, in fact, I would start from there), as our testsuite can
> > be run only in CMake builds.
> >
> I can do so, (locally switch to cmake) I have a little experience with
> cmake so I should be able to figure it out. From my distro point of
> view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
> ebuild, but it hasn't happened yet. Perhaps this could be expedited, I
> can contact the package maintainer.
FTR, CMake would probably see that we our host is neither x86_64 nor i386,
and won't build anything in compiler-rt. If it wouldn't, we should go and
fix this.
Maybe. Although, technically it *is* x86_64, but with a different ABI.
I don't know how that's handled, although it works for ARM/MIPS, right??
compiler-rt does make a convenient compiler test though..
fatal error: error in backend: Cannot select: 0x30ccf38: ch = brind
0x30cccf8:1, 0x30cc368 [ORD=109] [ID=11]
0x30cc368: i32 = add 0x30cccf8, 0x30cc518 [ORD=109] [ID=10]
0x30cccf8: i32,ch = load 0x30cc2d8:1, 0x30ccfc8,
0x30cc6c8<LD4[JumpTable]> [ORD=109] [ID=9]
0x30ccfc8: i32 = add 0x30cc5a8, 0x30cc518 [ORD=109] [ID=8]
0x30cc5a8: i32 = shl 0x30cc2d8, 0x30ccc68 [ORD=109] [ID=7]
0x30cc2d8: i32,ch = CopyFromReg 0x30a6168, 0x30cc908 [ORD=109]
[ID=5]
0x30cc908: i32 = Register %vreg29 [ID=1]
0x30ccc68: i8 = Constant<2> [ID=4]
0x30cc518: i32 = X86ISD::WrapperRIP 0x30ccab8 [ID=6]
0x30ccab8: i32 = TargetJumpTable<0> [ID=3]
0x30cc6c8: i32 = undef [ID=2]
0x30cc518: i32 = X86ISD::WrapperRIP 0x30ccab8 [ID=6]
0x30ccab8: i32 = TargetJumpTable<0> [ID=3]
In function: __atomic_compare_exchange
clang: error: clang frontend command failed with exit code 70 (use -v to
see invocation)
clang version 3.4 (trunk)
Target: x86_64-pc-linux-gnux32
Thread model: posix
This is what happens when compiling compiler-rt with -m64 or -mx32,
exact same error without my patches (where -mx32 is supported).
However, I know -mx32 isn't working since the objects always end up
elf_x86_64, which is why I known this is a somewhat unrelated bug.
I should add, with -m32, the same clang compiles atomic.c without error.
>
> > > Hi Steven,
> > >
> > > This looks interesting and raises a number of questions
> > >
> > > 1) Does applying this patch actually bring working sanitizers to x32
> > > platform?
> > > That is, after you build the clang, does "clang -fsanitize=whatever
> > foo.c"
> > > compile/link/run with expected results?
> > > I doubt that, as there is some platform-specific code in all the
> > > sanitizers, and many of them heavily depend
> > > on the address space layout. Porting TSan and MSan to 32-bit address
> > space
> > > is especially hard, and we don't plan
> > > to do this anytime soon. I think it makes sense to build only the
> > libraries
> > > that are expected to work on a given arch.
> > >
> > I should have made clear this is very much a WIP. I expect to have make
> > an effort to port compiler-rt, but first I need to be able to generate
> > elf32_x86_64 objects with Clang. I confess, my method of porting code
> > to x32 so far has consisted of trying to build existing code and
> > modifying what breaks, or disabling features where inapplicable. I
> > haven't studied the code well enough to understand and predict where
I'm
> > going to have make modification in advance. It's worked so far with
> > most things I've attempted but LLVM/Clang/compiler-rt is an order of
> > magnitude more complex than anything else I've poked at. The biggest
> > success I've had so far with this method is porting of Mozilla
> > Spidermonkey/js17.
> >
>
> So, IIUC, if you're working on making LLVM/Clang/compiler-rt source tree
> build on x32 host, I suggest your first step should be: disable building
> compiler-rt
> on x32 host One of the reasons: in makefile-based build compiler-rt
> sources
> are compiled with just-built Clang (I honestly don't know if Clang can
> target x32
> at the moment). Another reason: compiler-rt code is really arch-specific,
> and even
> if it compiles, it wouldn't work.
>
See accompanied Clang patch in other email. I've attempted to get Clang
to target x32, I'm also suspicious that x32 built clang isn't correctly
working to produce 64-bit code, but that's a somewhat different bug that
I haven't yet attempted to address. Without modifying Clang to target
x32, the x32 *built* Clang fails to compile compiler-rt with -m64, I
need help decoding the crash backtrace to figure out where that bug
lies, but it's not related to (or helped by) my patch.
So, it looks like Clang built on x32 host:
(a) doesn't work with -m64 (can't target x86_64)
(b) can target x32 with your patch.
If that's the case, (a) should be investigated (maybe, you can open a bug
for this
as a start), while you should try to get your patch for (b) submitted.
Let's move the
discussion to that thread, most likely you'll have to add tests and find a
reviewer for that patch.
>
> >
> > > 2) If we're stepping on the path of porting some sanitizers to x32,
it
> > > would be great to
> > > setup a buildbot, make our tests pass cleanly, and make this bot
public
> > to
> > > catch regressions.
> > > Do you have plans for that? Running sanitizer test suites also leads
us
> > > to...
> > >
> > I'm new to llvm/clang, I'm going to have to take guidance from others.
> >
> > > 3) Do you plan to add support for building sanitizers on x32 to CMake
> > build
> > > system? It would
> > > make sense (and, in fact, I would start from there), as our
testsuite can
> > > be run only in CMake builds.
> > >
> > I can do so, (locally switch to cmake) I have a little experience with
> > cmake so I should be able to figure it out. From my distro point of
> > view, the plan on Gentoo is to switch to CMake for the LLVM/Clang
> > ebuild, but it hasn't happened yet. Perhaps this could be expedited, I
> > can contact the package maintainer.
>
>
> FTR, CMake would probably see that we our host is neither x86_64 nor
i386,
> and won't build anything in compiler-rt. If it wouldn't, we should go and
> fix this.
>
Maybe. Although, technically it *is* x86_64, but with a different ABI.
I don't know how that's handled, although it works for ARM/MIPS, right??
Yes, we're able to target ARM on x86_64 host if that's what you're asking
about.
Actually, I was referring to supporting multiple ABIs under the same
ARCH. (o32, n32, n64 for MIPS, and EABI EABIHF etc for ARM) I need to
study the build system code a little more to figure out what it does,
unless someone can clarify..?
Actually, I've just fixed a bug where when running on a x32 host the
Environment part of the Triple is always GNUX32, I've now changed it so
that it gets properly set to GNU. This *fixes* the -m64 case, since
otherwise it *always* tries to compile to ILP32 except for some reason
the object ends up as elf_x86_64, not sure why yet... (can anybody help
here?) The -m32 case worked because then the Arch component was set to
i386 which is !=x86_64 as protects all the GNUX32 tests for throughout
the patch.
The upshot of this is the x86_64 target is now working on x32 hosts for
the first time!
Unfortunately, it does reveal a problem with X32; or possibly it's just
that atomic.c from compiler-rt won't work as is with X32 target, but it
shouldn't be triggering a crash...
Will send new patch shortly.