Build issues on Solaris

Hi all,

   I've encountered a couple of minor build issues on Solaris that have crept in since 2.5, fixes below:

   1. In lib/Target/X86/X86JITInfo.cpp, there is:

// Check if building with -fPIC
#if defined(__PIC__) && __PIC__ && defined(__linux__)
#define ASMCALLSUFFIX "@PLT"
#else
#define ASMCALLSUFFIX
#endif

Which causes a link failure due to the non-PLT relocation that results. I'm not sure if this should be
#if defined(__PIC__) && __PIC__ && (defined(__linux__) || defined(__sun__))

or

#if defined(__PIC__) && __PIC__ && !defined(__APPLE__)

or if it should be a configure test, which might be safer. Are there any x86 platforms (other than apple) that don't need PLT-indirect calls?

2. GenLibDeps.pl was changed in r73228 to remove the -g option from nm - this breaks with Sun nm, which needs the -g to emit output compatible with the regexps used in GenLibDeps.pl. Reverting this change works correctly (although it does print some warnings).

The patch had the comment:
     Fix the SUS usage of `nm` on Unix platforms such as Solaris or
     AuroraUX; -u is undefined symbols, and says "those external to each
     object file" so using -g would seem redundant, The error message was
     as follows: "/usr/xpg4/bin/nm: -e or -g set, -u ignored".

Actually it was the -u that's redundant - the -g was necessary for GenLibDeps.pl:

$ nm -p -u libLLVMCore.a
libLLVMCore.a[AsmWriter.o]:
     __asert
     atof
     _GLOBAL_OFFSET_TABLE_
     isalnum
     ... etc ...

$ nm -p -g libLLVMCore.a
libLLVMCore.a[AsmWriter.o]:
0000000000 U __assert
0000000000 U atof
0000000000 U _GLOBAL_OFFSET_TABLE_
0000000000 U isalnum

The latter being what GenLibDeps.pl expects to see. All else being equal though, it might be better to change GenLibDeps.pl to recognize the first version:
--- GenLibDeps.pl (revision 78653)
+++ GenLibDeps.pl (working copy)
@@ -100,7 +100,7 @@
      print " <dt><b>$lib</b</dt><dd><ul>\n";
    }
    open UNDEFS,
- "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' | sort | uniq |";
+ "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' -e 's/^ *//'

sort | uniq |";

    my %DepLibs;
    while (<UNDEFS>) {
      chomp;

Cheers,
Nathan

Good day Nathan,

If your using the stock GCC with Solaris, I am sorry to inform you its
broken, Please see:
http://llvm.org/docs/GettingStarted.html#brokengcc
Please be warned as your get miss compiles and things will do strange `stuff`.

As can be seen here, the reverse is true for both the GNU binutils and
XPG4 versions of `nm' :

-bash-3.2$ /usr/xpg4/bin/nm -p -u `find . -name libLLVMCore.a` | head

./Debug/lib/libLLVMCore.a[AsmWriter.o]:
0000000000 U __assert
0000000000 U __clzdi2
0000000000 U __udivdi3
0000000000 U __umoddi3
0000000000 U _GLOBAL_OFFSET_TABLE_
0000000000 U _ZdaPv
0000000000 U _ZdlPv
-bash-3.2$ /usr/bin/nm -p -u `find . -name libLLVMCore.a` | head

./Debug/lib/libLLVMCore.a[AsmWriter.o]:
    __assert
    __clzdi2
    __udivdi3
    __umoddi3
    _GLOBAL_OFFSET_TABLE_
    _ZdaPv
    _ZdlPv

-bash-3.2$ /usr/bin/nm -p -g `find . -name libLLVMCore.a` | head

./Debug/lib/libLLVMCore.a[AsmWriter.o]:
0000000000 U __assert
0000000000 U __clzdi2
0000000000 T __i686.get_pc_thunk.bx
0000000000 T __i686.get_pc_thunk.cx
0000000000 U __udivdi3
0000000000 U __umoddi3
0000000000 U _GLOBAL_OFFSET_TABLE_
-bash-3.2$ /opt/gcc4/bin/nm -p -g `find . -name libLLVMCore.a` | head

AsmWriter.o:
00000000 T __i686.get_pc_thunk.cx
         U _GLOBAL_OFFSET_TABLE_
00000000 W _ZnwjPv
00000000 W _ZN4llvm24AssemblyAnnotationWriter17emitFunctionAnnotEPKNS_8FunctionERNS_11raw_ostreamE
00000000 W _ZN4llvm24AssemblyAnnotationWriter24emitBasicBlockStartAnnotEPKNS_10BasicBlockERNS_11raw_ostreamE
00000000 W _ZN4llvm24AssemblyAnnotationWriter22emitBasicBlockEndAnnotEPKNS_10BasicBlockERNS_11raw_ostreamE
00000000 W _ZN4llvm24AssemblyAnnotationWriter20emitInstructionAnnotEPKNS_11InstructionERNS_11raw_ostreamE
00000000 W _ZNK4llvm12PATypeHandlecvPNS_4TypeEEv
-bash-3.2$
-bash-3.2$
-bash-3.2$
-bash-3.2$ /usr/bin/nm -p -u `find . -name libLLVMCore.a` | head

./Debug/lib/libLLVMCore.a[AsmWriter.o]:
    __assert
    __clzdi2
    __udivdi3
    __umoddi3
    _GLOBAL_OFFSET_TABLE_
    _ZdaPv
    _ZdlPv
-bash-3.2$ /opt/gcc4/bin/nm -p -u `find . -name libLLVMCore.a` | head

AsmWriter.o:
         U _GLOBAL_OFFSET_TABLE_
         U _ZTVN4llvm11raw_ostreamE
         U _ZTVN4llvm14raw_os_ostreamE
         U _ZTVN4llvm18raw_string_ostreamE
         U _ZNKSs4sizeEv
         U _ZNKSsixEj
         U _ZNKSs5c_strEv
         U _ZNKSs6lengthEv

I believe this is a bug in the version of `nm` your using, prob /usr/bin/nm .

I recommend configuring your PATH differently and using the GCC from:

( Installs to /opt/gcc4 )

Cheers,
Edward O'Callaghan.

Hello, Nathan

or if it should be a configure test, which might be safer. Are there
any x86 platforms (other than apple) that don't need PLT-indirect calls?

Yes, mingw. However just tweaking the define is not enough - we're not
loading address of GOT into ebx before the call (on 32 bit ABIs) thus
the call will be to nowhere.

Good day Nathan,

If your using the stock GCC with Solaris, I am sorry to inform you its
broken, Please see:
http://llvm.org/docs/GettingStarted.html#brokengcc
Please be warned as your get miss compiles and things will do strange `stuff`.

Hi Edward,

    Good to know - I think I've only built 32-bit versions with 3.4.3, so I haven't run into this particular bug there. I usually do the optimized and 64-bit builds with gcc 4.2 or 4.3.

As can be seen here, the reverse is true for both the GNU binutils and
XPG4 versions of `nm' :

<snip nm examples>

I believe this is a bug in the version of `nm` your using, prob /usr/bin/nm .

Yes, Sun nm (/usr/bin/nm) produces different output to GNU nm (and the XPG4 version is different again, yes), and always has as far as I know - this isn't a bug in nm per se. In any case LLVM 2.4 and 2.5 built correctly with /usr/bin/nm, so this seems like a regression to me. As I said, the best fix is probably to adjust the sed expression so that it can handle the output from /usr/bin/nm -p -u (or any other nm we know about).

Cheers,
Nathan

Hi Nathan,

I said, the best fix is probably to adjust the sed expression so that it can handle the output from /usr/bin/nm -p -u (or any other nm we know about).

I agree that the sed change is probably best.

Ciao,

Duncan.

Hi Nathan,

The latter being what GenLibDeps.pl expects to see. All else being equal though, it might be better to change GenLibDeps.pl to recognize the first version:
--- GenLibDeps.pl (revision 78653)
+++ GenLibDeps.pl (working copy)
@@ -100,7 +100,7 @@
      print " <dt><b>$lib</b</dt><dd><ul>\n";
    }
    open UNDEFS,
- "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' | sort | uniq |";
+ "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' -e 's/^ *//' > sort | uniq |";
    my %DepLibs;
    while (<UNDEFS>) {
      chomp;

starting at line 123 there seems to be a by-hand version of the
sed command. I guess this needs to be adjusted too. Probably
sed can be used nowadays.

Ciao,

Duncan.

Duncan Sands wrote:

Hi Nathan,

The latter being what GenLibDeps.pl expects to see. All else being equal though, it might be better to change GenLibDeps.pl to recognize the first version:
--- GenLibDeps.pl (revision 78653)
+++ GenLibDeps.pl (working copy)
@@ -100,7 +100,7 @@
      print " <dt><b>$lib</b</dt><dd><ul>\n";
    }
    open UNDEFS,
- "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' | sort | uniq |";
+ "$nmPath -u $Directory/$lib | sed -e 's/^[ 0]* U //' -e 's/^ *//' > sort | uniq |";
    my %DepLibs;
    while (<UNDEFS>) {
      chomp;
    
starting at line 123 there seems to be a by-hand version of the
sed command. I guess this needs to be adjusted too. Probably
sed can be used nowadays.
  

I'd adjust it to be able to run in the first place (attaching patch).

It's intended to run exactly when nm fails to find dependencies, so die'ing out is pointless before it runs. If allowed to run, this supports Perls whose native C system command shell is cmd.exe: Strawberry Perl, ActivePerl, and any Perl built for Windows *other* than the MingW32 pre-built binaries. (cmd.exe chokes on the pipe into sed, causing the whole construct to fail.)

If the first nm finds any dependencies at all, the by-hand version won't run. It's currently optimized for MingW32 nm.

Kenneth

GenLibDeps.pl.patch (599 Bytes)

Ok, so configure might be the way to go then, maybe something like the following? I haven't tested this on mingw, but it looks to do the right thing on Solaris, Linux + OS X.

Index: lib/Target/X86/X86JITInfo.cpp

Hello, Nathan

loading address of GOT into ebx before the call (on 32 bit ABIs) thus
the call will be to nowhere.

Good point, I didn't look closely enough at the calling sequence. I assume
this has to be broken on Linux/x86 at the moment too? I've done up a quick
and dirty implementation below for the sake of discussion, which compiles
(and doesn't break lli), but hasn't been tested beyond that point as yet.
Admittedly this adds even more versions of X86CompilationCallback, but the
mixed version with #ifs seemed to be pretty hard to follow.

There was some PR wrt this issue (sorry, I remembered about it only
now). It was decided that proper way of doing things is just declare
the compilation callback as static - then you won't need to assemble
call via PLT at all. There was even patch for that, however I was not
working for me for some weird reason and I had no time to dig, why.

You know, now that you mention it why _are_ there two versions of this bit? If there's a concern about sed possibly being broken, we could just remove sed altogether and let perl handle it:

Index: GenLibDeps.pl

Nathan Keynes wrote:

You know, now that you mention it why _are_ there two versions of this bit? If there's a concern about sed possibly being broken, we could just remove sed altogether and let perl handle it:

Perl uses whatever command shell that its C compiler uses. If that command shell happens to be a Bourne shell, then the whole pipe sequence works (subject to availability of sed, sort, and uniq but that is pretty much a freebie on systems with Bourne shells) and should be used to avoid the completely unnecessary risk of breaking pre-existing platforms. (I defer to those who actually have seniority regarding whether that risk is completely unnecessary, but that is what dictated my strategy in implementing this originally.)

If the C command shell is cmd.exe or command.com (which is true for ActivePerl, Strawberry Perl, and Perl built from tarball using MingW32, but *not* the binary package of Perl distributed by the MingW project), the pipe from nm to sed fails. Nothing is detected regardless of the results from nm in this case.

Kenneth

Ah, PR3801 I think, and yes that would be the far better way of fixing this. Static looks like it will work properly here, but I notice that the PR's patch adds the anonymous namespace rather than static. I'm not sure whether or not that will work in practice, but in testing it seems that gcc 4.3 at least still generates PLT linkage for non-static functions in the anonymous namespace[0].

Cheers,
Nathan

[0] Which is what I would have expected, actually.