Problems loading passes on Mac OS X

Hi, I can't get opt to list (in -help) passes that I load using -load.
I see in the list archives that a similar problem has been brought up before but I didn't see whether it was resolved. Also, this is on Mac OS X, and the previous question was about Linux.

This problem happens with the Hello pass, so I'll use that to illustrate. I'm using LLVM 1.2.
The plugin loads with no complaints, so dlopen() is returning success, and OS X's dlcompat probably isn't the problem.
However, it never gets registered. I've included a transcript below.

I also included some info about the compiler versions, etc. - It's OS X 10.3.4.
Note that on OS X, 'libtool' isn't gnu libtool - that's called 'glibtool', if that matters.

Does anyone have some hints on how to get this working? Thanks.

transcript:

% cd llvm/lib/Transforms/Hello/
% make
Compiling Hello.cpp
Linking hello dynamic debug library

Hi, I can't get opt to list (in -help) passes that I load using -load.
I see in the list archives that a similar problem has been brought up
before but I didn't see whether it was resolved. Also, this is on Mac
OS X, and the previous question was about Linux.

The problem is that this has not been implemented yet on Mac OSX. If
you're familiar with the OS/X interfaces for loading .so files, it should
be pretty easy. Take a look at lib/Support/DynamicLinker.cpp. I think
that OS/X doesn't have windows.h :), and it also doesn't have dlopen. If
you can provide the necessary magic for your platform, it should just
work.

-Chris

This problem happens with the Hello pass, so I'll use that to
illustrate. I'm using LLVM 1.2.
The plugin loads with no complaints, so dlopen() is returning success,
and OS X's dlcompat probably isn't the problem.
However, it never gets registered. I've included a transcript below.

I also included some info about the compiler versions, etc. - It's OS X
10.3.4.
Note that on OS X, 'libtool' isn't gnu libtool - that's called
'glibtool', if that matters.

Does anyone have some hints on how to get this working? Thanks.

transcript:

% cd llvm/lib/Transforms/Hello/
% make
Compiling Hello.cpp
Linking hello dynamic debug library
----------------------------------------------------------------------
Libraries have been installed in:
    /Users/mike/Documents/hpcl/LLVM/llvm/lib/Debug

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
    - add LIBDIR to the `DYLD_LIBRARY_PATH' environment variable
      during execution

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
======= Finished building hello dynamic debug library =======
Linking hello.o

% opt -load ../../Debug/libhello.dylib -hello
Unknown command line argument '-hello'. Try: 'opt --help'

% g++ -v
Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
Thread model: posix
gcc version 3.3 20030304 (Apple Computer, Inc. build 1495)

% glibtool --version
ltmain.sh (GNU libtool) 1.5 (1.1220 2003/04/05 19:32:58)

_______________________________________________
LLVM Developers mailing list
LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev

-Chris

Unfortunately it's not that easy, although I'd love to fix it if I can figure out how.

OS X 10.3 does have dlfcn.h and dlopen() - for 10.3, they added the dlcompat library that uses the OS X NSLink* stuff to support dlopen and dlsym. configure seems to pick that up fine.

It seems like the code that's there already should work fine, so I'm not sure where else to look.
I'm not a C++ guru, so if there's C++ linking difficulties or something I'm not sure I'll be able to figure that out without help.

I used the following short code to test loading the .dylib, and dlopen works fine, but dlsym is less pretty - on OS X, it automatically prepends an underscore to the symbol, which means that if you are looking for main(), you ask for "main" and it looks for "_main", which is how the symbols are named in Mach-O.
This could be the source of a problem - how does GetAddressOfSymbol in DynamicLinker.cpp use dlsym?
Does it assume anything about the symbol names?

#include <dlfcn.h> // dlcompat
int main(int argc, const char **argv){
   void *obj = dlopen(argv[1], RTLD_NOW | RTLD_GLOBAL);
   void * sym = dlsym(obj,argv[2]);
   printf("done: obj was %x sym was %x %s\n", obj, sym , dlerror());
   return 0;
}

-mike

I used the following short code to test loading the .dylib, and dlopen
works fine, but dlsym is less pretty - on OS X, it automatically
prepends an underscore to the symbol, which means that if you are
looking for main(), you ask for "main" and it looks for "_main", which
is how the symbols are named in Mach-O.
This could be the source of a problem - how does GetAddressOfSymbol in
DynamicLinker.cpp use dlsym?

I believe that GetAddressOfSymbol is only used by the JIT. The -load
mechanism just dlopens a shared object. Shared objects for plugins have
static constructors (".init" section) that are supposed to autoregister
the pass when the plugin is loaded.

I don't know anything about how dynamic object loading works on OS/X
(obviously), but it sounds like it's not running the static constructor.

Does it assume anything about the symbol names?

I don't know what it will take to get the JIT working on OS/X: it might
have to autoprepend an _. Since we don't (yet) have a PPC JIT, this isn't
an issue though.

-Chris

OK, seems like I've got some more looking to do.
From what I can find, the only difference with OS X and dynamic libraries is that you can't depend on the order of calling static constructors, but not that you can't depend on them getting called. :slight_smile:

If I find that library handling in LLVM needs changes to work with OS X, I'll send a patch.

Thanks,
-mike

Solved.

Short version:

You don't need to change DynamicLinker.cpp at all. dlopen() should just work, except that the defaults for libtool are to produce the wrong kind of shared object.

On OS X, libtool needs to be passed the -module option to create dlopen-able objects:

% diff Makefile.rules.old Makefile.rules
326c326
< Link := $(LIBTOOL) --mode=link $(CXX)

Solved.

Yaay!

Short version:

You don't need to change DynamicLinker.cpp at all. dlopen() should just
work, except that the defaults for libtool are to produce the wrong
kind of shared object.

Okay.

On OS X, libtool needs to be passed the -module option to create
dlopen-able objects:

% diff Makefile.rules.old Makefile.rules
326c326
< Link := $(LIBTOOL) --mode=link $(CXX)
---
> Link := $(LIBTOOL) --mode=link $(CXX) -module

This hack isn't a complete fix, because SHLIBEXT is still set wrong.
(It should be .so not .dylib). I couldn't figure out how to do that
right, so I didn't try and it didn't matter for getting it running,
although it's clear some make targets are messed up.

Okay, I'm glad that this is working for you and that you found the root
cause of the problem. I will defer to one of our autoconf/libtool experts
on how to get this implemented correctly and portably. In particular, it
seems like the incorrect detection of SHLIBEXT could be a blocker until it
is fixed.

-Chris

Elaboration:

not passing -module makes libtool use 'g++ -dynamiclib', which creates
Mach-O MH_DYLIB files, that only have static ctors called if a symbol
in that same file is used. (I got it to work by dlsym'ing the mangled
ctor's name, but that's not a solution.)

if you pass -module, libtool uses 'g++ -bundle', which creates Mach-O
MH_BYNDLE files, that work as dlopen() expects.

Apparently the reason there are these two types of Mach-O files is that
you can use a .dylib with a static linker as well as being loaded. I
guess this isn't a big deal when using libtool anyway.

(End Elaboration)

So, I'm set to work, and hopefully this gives you enough info to add on
OS X plugin loading support. I'm a little too tired now to try to
figure out exactly how to set up configure.ac and Makefile.rules, but
I'm hoping someone else will know already and it'll be easy.

Let me know if I can be of any more help for this, and thanks for your
help!
-mike

> OK, seems like I've got some more looking to do.
> From what I can find, the only difference with OS X and dynamic
> libraries is that you can't depend on the order of calling static
> constructors, but not that you can't depend on them getting called. :slight_smile:
>
> If I find that library handling in LLVM needs changes to work with OS
> X, I'll send a patch.
>
> Thanks,
> -mike
>
>
>>> I used the following short code to test loading the .dylib, and
>>> dlopen
>>> works fine, but dlsym is less pretty - on OS X, it automatically
>>> prepends an underscore to the symbol, which means that if you are
>>> looking for main(), you ask for "main" and it looks for "_main",
>>> which
>>> is how the symbols are named in Mach-O.
>>> This could be the source of a problem - how does GetAddressOfSymbol
>>> in
>>> DynamicLinker.cpp use dlsym?
>>
>> I believe that GetAddressOfSymbol is only used by the JIT. The -load
>> mechanism just dlopens a shared object. Shared objects for plugins
>> have
>> static constructors (".init" section) that are supposed to
>> autoregister
>> the pass when the plugin is loaded.
>>
>> I don't know anything about how dynamic object loading works on OS/X
>> (obviously), but it sounds like it's not running the static
>> constructor.
>>
>>> Does it assume anything about the symbol names?
>>
>> I don't know what it will take to get the JIT working on OS/X: it
>> might
>> have to autoprepend an _. Since we don't (yet) have a PPC JIT, this
>> isn't
>> an issue though.
>>
>> -Chris
>>
>>
>>
>>>
>>>>
>>>>> Hi, I can't get opt to list (in -help) passes that I load using
>>>>> -load.
>>>>> I see in the list archives that a similar problem has been brought
>>>>> up
>>>>> before but I didn't see whether it was resolved. Also, this is on
>>>>> Mac
>>>>> OS X, and the previous question was about Linux.
>>>>
>>>> The problem is that this has not been implemented yet on Mac OSX.
>>>> If
>>>> you're familiar with the OS/X interfaces for loading .so files, it
>>>> should
>>>> be pretty easy. Take a look at lib/Support/DynamicLinker.cpp. I
>>>> think
>>>> that OS/X doesn't have windows.h :), and it also doesn't have
>>>> dlopen.
>>>> If
>>>> you can provide the necessary magic for your platform, it should
>>>> just
>>>> work.
>>>>
>>>> -Chris
>>>>
>>>>
>>>>
>>>>
>>>>> This problem happens with the Hello pass, so I'll use that to
>>>>> illustrate. I'm using LLVM 1.2.
>>>>> The plugin loads with no complaints, so dlopen() is returning
>>>>> success,
>>>>> and OS X's dlcompat probably isn't the problem.
>>>>> However, it never gets registered. I've included a transcript
>>>>> below.
>>>>>
>>>>> I also included some info about the compiler versions, etc. - It's
>>>>> OS
>>>>> X
>>>>> 10.3.4.
>>>>> Note that on OS X, 'libtool' isn't gnu libtool - that's called
>>>>> 'glibtool', if that matters.
>>>>>
>>>>> Does anyone have some hints on how to get this working? Thanks.
>>>>>
>>>>> transcript:
>>>>>
>>>>> % cd llvm/lib/Transforms/Hello/
>>>>> % make
>>>>> Compiling Hello.cpp
>>>>> Linking hello dynamic debug library
>>>>> -------------------------------------------------------------------
>>>>> ---
>>>>> Libraries have been installed in:
>>>>> /Users/mike/Documents/hpcl/LLVM/llvm/lib/Debug
>>>>>
>>>>> If you ever happen to want to link against installed libraries
>>>>> in a given directory, LIBDIR, you must either use libtool, and
>>>>> specify the full pathname of the library, or use the `-LLIBDIR'
>>>>> flag during linking and do at least one of the following:
>>>>> - add LIBDIR to the `DYLD_LIBRARY_PATH' environment variable
>>>>> during execution
>>>>>
>>>>> See any operating system documentation about shared libraries for
>>>>> more information, such as the ld(1) and ld.so(8) manual pages.
>>>>> -------------------------------------------------------------------
>>>>> ---
>>>>> ======= Finished building hello dynamic debug library =======
>>>>> Linking hello.o
>>>>>
>>>>> % opt -load ../../Debug/libhello.dylib -hello
>>>>> Unknown command line argument '-hello'. Try: 'opt --help'
>>>>>
>>>>> % g++ -v
>>>>> Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
>>>>> Thread model: posix
>>>>> gcc version 3.3 20030304 (Apple Computer, Inc. build 1495)
>>>>>
>>>>> % glibtool --version
>>>>> ltmain.sh (GNU libtool) 1.5 (1.1220 2003/04/05 19:32:58)
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>
>>>>
>>>> -Chris
>>>>
>>>> --
>>>> http://llvm.cs.uiuc.edu/
>>>> http://www.nondot.org/~sabre/Projects/
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
>>> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>
>> -Chris
>>
>> --
>> http://llvm.cs.uiuc.edu/
>> http://www.nondot.org/~sabre/Projects/
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
>> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
> http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev
>

_______________________________________________
LLVM Developers mailing list
LLVMdev@cs.uiuc.edu http://llvm.cs.uiuc.edu
http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev

-Chris