ILP32, -arch i386 and x86_64 Apple systems

Hi Everyone,

(Re-asking on CFE-Dev). We took a bug report on some line assembly
code (https://github.com/weidai11/cryptopp/issues/72). I have two
questions at the end on ILP32, Clang and Apple.

The code that took the bug performs the following:

        asm volatile
        (
            // save ebx in case -fPIC is being used
# if BOOL_X32 || BOOL_X64
            "pushq %%rbx; cpuid; mov %%ebx, %%edi; popq %%rbx"
# else
            "push %%ebx; cpuid; mov %%ebx, %%edi; pop %%ebx"
# endif
            : "=a" (output[0]), "=D" (output[1]), "=c" (output[2]),
"=d" (output[3])
            : "a" (input), "c" (0)
        );

The BOOL_* defines are mutually exclusive. BOOL_X32 is set when
__ILP32__ is defined. BOOL_X86 is defined when __i386__ (and friends)
is defined. BOOL_X64 is defined when __x86_64__ (and friends) is
defined. We set the defines based on " System V Application Binary
Interface, AMD64 (With LP64 and ILP32 Programming Models)" [1].

The compiler error is:

    $ make cpu.o
    clang++ -DNDEBUG -g2 -O2 -arch i386 -fPIC -march=native -pipe -c cpu.cpp
    cpu.cpp:104:4: error: register %rbx is only available in 64-bit mode
                            "pushq %%rbx; cpuid; mov %%ebx, %%edi; popq %%rbx"
                           ^
    <inline asm>:1:8: note: instantiated into assembly here
           pushq %rbx; cpuid; mov %ebx, %edi; popq %rbx
              ^~~~~
    cpu.cpp:104:4: error: register %rbx is only available in 64-bit mode
                           "pushq %%rbx; cpuid; mov %%ebx, %%edi; popq %%rbx"
                            ^
    <inline asm>:1:42: note: instantiated into assembly here
           pushq %rbx; cpuid; mov %ebx, %edi; popq %rbx
                                                    ^~~~
    2 errors generated.

It appears Clang sets ILP32 related defines when using -arch i386 on x86_64:

    $ clang++ -arch i386 -dM -E - < /dev/null | egrep -i "(86|64|ilp)"
    #define _ILP32 1
    #define __ILP32__ 1
    ...
    #define __i386 1
    #define __i386__ 1
    #define i386 1

As far as I know, when using ILP32 on x86_64, we must interact with
the stack using 64-bit values and registers.

My first question is, is _ILP32_ and _i386_ supposed to be defined
together? According to the docs I found, its not an expected/valid
configuration. _ILP32_ and _x86_64_ is an expected/valid
configuration.

My second question is, how should I work around this? Should I special
case Clang on Apple and attempt to push a 32-bit value? Or should I do
something else?

Thanks in advance.

I’m not sure why you would expect otherwise. i386 is an ILP32 target, so would expect to have ILP32-related defines (or, if they are not globally available, would expect some operating systems to use them). If you are using ILP32 to mean x32, then this sounds like a bug in your code.

David

My first question is, is _ILP32_ and _i386_ supposed to be defined
together? According to the docs I found, its not an expected/valid
configuration. _ILP32_ and _x86_64_ is an expected/valid
configuration.

I’m not sure why you would expect otherwise.

Because of the documents covering the topic. They are cited at the
tail of the post.

... If you are using ILP32 to mean x32, then this sounds like a bug in your code.

OK, thanks.

When the i386 code is run on an x86_64 system (i.e., "arch i386
myprog.exe"), then we must interact with the stack using 64-bit values
and registers. Its an ABI requirement.

How can we do it if Clang's integrate assembler won't allow us to use
64-bit registers?

Jeff

When i386 code is run on an x86-64 system, it is using a different system call layer, but is otherwise a normal 386 extended mode environment. Without an explicit switch to long mode, it can not use any of the long mode registers. There is most definitely not an ABI requirement that i386 code use non-i386 instructions.

David

Thanks David.

I just checked a 64-bit Fedora system using the -m32 option (which I
believe is equivalent to -arch i386; corrections please). Neither GCC
nor Clang define ILP32 and friends. So this appears to be limited to
Apple systems.

As for the lack of an ABI requirement, I'm not sure... I've read the
relevant docs, and I don't see the exception you are discussing (i.e.,
not an ABI requirement). I think part of the problem is its under
specified by Apple, and deferring to the System V ABI docs is not the
best strategy since they are not following the docs they are deferring
to.

Jeff

My bad... I forgot the "-i" in my grep. It appears the issue is
limited to Clang. GCC and Intel compilers do not define it:

fedora-22 $ clang++ -x c++ -m32 -dM -E - < /dev/null | egrep -i "(ilp)"
#define _ILP32 1
#define __ILP32__ 1

fedora-22 $ g++ -x c++ -m32 -dM -E - < /dev/null | egrep -i "(ilp)"
$

ubuntu-14 $ /opt/intel/bin/icpc -x c++ -m32 -dM -E - < /dev/null |
egrep -i "(ilp)"
$

I'm not sure about Comeau's compiler because I do not have access to it.

Jeff

Right. Clang’s logic for defining the ILP macros is very simple:

if (TI.getPointerWidth(0) == 64 && TI.getLongWidth() == 64
&& TI.getIntWidth() == 32) {
Builder.defineMacro("_LP64");
Builder.defineMacro(“LP64”);
}

if (TI.getPointerWidth(0) == 32 && TI.getLongWidth() == 32
&& TI.getIntWidth() == 32) {
Builder.defineMacro("_ILP32");
Builder.defineMacro(“ILP32”);
}

It happens on all platforms, not just x86. I think this is reasonable behavior and we should keep it, even if x86 gcc -m32 doesn’t define this macro. As David said, don’t use ILP32 to detect x32. You’ll have a bad time on arm32.

Thanks. I also saw H.J. Lu's discussion related to ILP32 and
ARM64/AARCH64 on LKML

If you don't mind helping with a strategy, how should I check for X32?
The System V ABI only provides ILP32.

Thanks in advance.

Jeff

On my Debian Linux box, with gcc 4.9.2:

$ gcc -mx32 -dM -E -x c /dev/null | sort | grep -E "LP|amd64|i386|x86_64"
#define __amd64 1
#define __amd64__ 1
#define __ILP32__ 1
#define _ILP32 1
#define __x86_64 1
#define __x86_64__ 1

With clang 3.7.0:

$ clang -mx32 -dM -E -x c /dev/null | sort | grep -E "LP|amd64|i386|x86_64"
#define __amd64 1
#define __amd64__ 1
#define __ILP32__ 1
#define _ILP32 1
#define __x86_64 1
#define __x86_64__ 1

I get similar results on FreeBSD. So you could check as follows:

#if defined __x86_64__ && defined __ILP32__

-Dimitry

Ignoring for a moment that there is a pretty much standard header
covering cpuid, the 64bit alignment of the stack is a non-issue. Like
the compiler, just push two registers and pop two registers.

Joerg

Hi Everyone,

(Re-asking on CFE-Dev). We took a bug report on some line assembly
code (https://github.com/weidai11/cryptopp/issues/72). I have two
questions at the end on ILP32, Clang and Apple.

The code that took the bug performs the following:

        asm volatile
        (
            // save ebx in case -fPIC is being used
# if BOOL_X32 || BOOL_X64
            "pushq %%rbx; cpuid; mov %%ebx, %%edi; popq %%rbx"
# else
            "push %%ebx; cpuid; mov %%ebx, %%edi; pop %%ebx"
# endif
            : "=a" (output[0]), "=D" (output[1]), "=c" (output[2]),
"=d" (output[3])
            : "a" (input), "c" (0)
        );

The BOOL_* defines are mutually exclusive. BOOL_X32 is set when
__ILP32__ is defined. BOOL_X86 is defined when __i386__ (and friends)
is defined. BOOL_X64 is defined when __x86_64__ (and friends) is
defined. We set the defines based on " System V Application Binary
Interface, AMD64 (With LP64 and ILP32 Programming Models)" [1].

Since no-one else has explicitly pointed this out: this is the wrong
document. When you're targeting i386, the contents of the x86_64 psABI
document are irrelevant, since it is not the ABI document for your
platform. When targeting x86-64, we'll define __ILP32__ as specified in
that document (and it's a bug if we deviate from that), but when targeting
i386, we get to define it however we think is best (and it makes a lot of
sense to define it, because i386 is an ILP32 target) since it's a reserved
identifier.

The compiler error is: