Shared libs?

I've been playing with LLVM for a couple of months now and really am enjoying myself. I finally got around to doing something useful and had started implementing bindings for Ruby. I've been working off the tip of trunk and realized my build didn't have shared libs. I've spent the better part of a day trying to build llvm with shared libraries with no luck on either Mac OS X 10.5.2 , or Ubuntu 8.04.

Exactly what magic incantation must I pass to configure?

I've tried:

--enable-shared
--enable-shared=yes
--enable-shared=core, --enable-shared=Core, --enable-shared=all

and all of the above with various flags such as:

--enable-pic
--disable-static

And still not a single shared lib in sight. The only other flags I am using are "--enable-targets=host-only" and "--enable-optimized."

Tomy

This isn't first-hand, but from what I remember hearing on IRC,
putting llvm into shared libraries caused a ridiculous explosion in
dynamic linking (and therefore startup) times. So there is no option
to make shared libraries, at least at the moment.

If you'd like to elaborate on why exactly you need shared libs, you
might get a better answer.

-Eli

Eli Friedman wrote:

This isn't first-hand, but from what I remember hearing on IRC,
putting llvm into shared libraries caused a ridiculous explosion in
dynamic linking (and therefore startup) times. So there is no option
to make shared libraries, at least at the moment.

Well, by tweaking configure and make options, I've managed to build LLVM
2.2 shared libraries on Linux (x86), and they seemed to work fine. (I
did have to work around a circular dependencies error, though.)

This makes a *lot* of sense. E.g., my application is a functional
language interpreter (http://pure-lang.sf.net) which has to be linked
against most of LLVM, and it makes a whole lot of a difference to have
those 8+MB of libraries being handled by the dynamic linker instead of
including them all in a fat interpreter executable, for the same reason
that you don't want each and every C/C++ application to link libc and
the C++ library statically. Just imagine that you have 50 instances of
the interpreter running simultaneously!

Note that you can also work around this by just linking the static LLVM
libs into a shared lib which in turn is linked against your application.
That's what I actually do with my interpreter now, since the users
shouldn't have to tweak the LLVM stuff just to build my interpreter.
(Unfortunately, that approach doesn't work on x86-64 with LLVM 2.2,
since some parts of the LLVM JIT apparently contain non-relocatable
code; I hope that this will be fixed in the forthcoming LLVM 2.3.)

Albert

Because I'm writing an extension to a scripting language interpreter. Perl/Python/Ruby all need to be able to dynamically load the library when the user 'requires' or 'imports' the extension.

Obviously I can write a C wrapper which references LLVM and links it as a shared object. But it might be nice to issue a warning when someone uses the "--enable-shared" flag, or perhaps remove it if it is not supported. Or at least put something on the web site that Google would find. :slight_smile:

Tomy

Thomas Hudson wrote:

If you'd like to elaborate on why exactly you need shared libs, you
might get a better answer.

Because I'm writing an extension to a scripting language interpreter.
Perl/Python/Ruby all need to be able to dynamically load the library
when the user 'requires' or 'imports' the extension.

Obviously I can write a C wrapper which references LLVM and links it
as a shared object. But it might be nice to issue a warning when
someone uses the "--enable-shared" flag, or perhaps remove it if it is
not supported. Or at least put something on the web site that Google
would find. :slight_smile:

"--enable-shared" is dangerous because no one really has thought out the
shared library structure. That's not to say that shared libraries are
evil, but as Chris Lattner pointed out, they can be too finely grained
(every library is a shared library) or too coarsely grained (LLVM is
nothing but a shared library). But if you're looking to integrate into a
scripting language, generating a shared library needs to be there. How
much or how little LLVM you need is _the_ question that needs answering.
For example, clearly, you wouldn't want the SPARC or Arm or MIPS code
generators polluting your shared library load space at runtime, but the
infrastructure to load code generators on-demand isn't there.

-scooter
"And swiftly and software, the dungeon master said, 'Roll for
initiative.'" :slight_smile:

Unfortunately it's not fixed in 2.3 :frowning: I made a patch ([1]) for 2.2 and gave
it to one of the developer, I guess he forgot about it, unfortunately it
doesn't apply on 2.3. Once I have the time (and a 64bits linux under the
hand), I will make a patch for 2.3 and submit it to llvm's bug system (unless
someone else want to do it before that :wink: )

[1] http://www.opengtl.org/download/X86JITInfo.cpp.pic.patch

Hello, Cyrille

Unfortunately it's not fixed in 2.3 :frowning: I made a patch ([1]) for 2.2 and gave
it to one of the developer, I guess he forgot about it

No, I don't forget about it.

The problem is that stuff for x86-64 seems to be really easier, than for
x86-32 (RIP-relative PIC vs GOT-relative, also we should think about
darwin). I thought to add x86-32 support there as well, but I was really
lack of free time, sorry.

Cyrille Berger wrote:

Unfortunately it's not fixed in 2.3 :frowning:

That's indeed unfortunate. On x86-64 the Pure interpreter currently is a
7MB behemoth, and most of that is LLVM. :wink: On 32 bit I have all that
stuff in a separate runtime library, resulting in a 27K interpreter
executable. It goes without saying that this makes a world of a
difference. I don't care if LLVM is a shared library itself, and I
understand the reasons speaking against that, but it should at least be
linkable into a dll.

[1] http://www.opengtl.org/download/X86JITInfo.cpp.pic.patch

Cool, many thanks! That's exactly what I've been looking for. :slight_smile: If you
do get around updating the patch for 2.3, it would be very kind if you
could let me know.

Thanks,
Albert

Hi Cyrille,

concerning your patch,

[1] http://www.opengtl.org/download/X86JITInfo.cpp.pic.patch

there are two minor glitches: s/#ifdef/#if/, s/#end/#endif/.

Otherwise it works great over here! Pure builds fine with a shared
runtime lib on 64 bit now, so I'm a happy camper. :slight_smile: I would like to
make this patch available on the Pure website, ok with you?

Albert

Hi,

Hi,

Cool, many thanks! That's exactly what I've been looking for. :slight_smile: If you
do get around updating the patch for 2.3, it would be very kind if you
could let me know.

I have updated the patch for llvm 2.3, and open a bug report to follow the
status on this subject: 2514 – Relocation when using llvm in shared library

Cyrille Berger wrote:

I have updated the patch for llvm 2.3, and open a bug report to follow the
status on this subject: 2514 – Relocation when using llvm in shared library

Great, many thanks! So now I just have to make Pure compile with LLVM
2.3. :slight_smile: I see in the release notes that some classes have changed
(notably the LLVMBuilder stuff), is there a porting guide available
somewhere?

Albert