To APR Or Not To APR. That is the question.

Folks,

I'm bringing the APR (Apache Portable Runtime) up for consideration as
the implementation of lib/System again. For those not familiar with it,
APR is the portability library used by the Apache Group and others in
open source projects such as HTTPD and Subversion. While its relatively
new, it is quickly gaining mind share and it provides lots of features
and supports numerous platforms today. For more details you can check
out: http://apr.apache.org/.

Our use of APR would be limited to the lib/System library. That is, we
would provide very thin C++ veneers on top of APR's C interface that
match the current lib/System and lib/Support APIs.

Since an APR implementation would get lib/System "done" on all our
platforms much faster than the current approach, I'd like to know if
anyone has any objections. Briefly, here's the upside/downside:

Upside:
      * Get lib/System implemented on all platforms faster
      * Avoid maintaining complicated platform independence code (the
        gust of lib/System) so that our attention is not diverted from
        compiler tool building.
      * Faster bug fixing in this area because other developers are
        working on it.
      * Faster support for new platforms because other developers are
        working on it.

Downside:
      * Makes LLVM dependent on a third party library
      * Makes LLVM platform support dependent on
      * Error handling in APR is somewhat strange and it could be quite
        difficult for us to continue to meet the "throw std::string"
        approach we have today.

Silence on this issue will be taken as tacit approval. If you have any
objections/thoughts/questions, please post them so we can discuss.

Thanks,

Reid.

I vote against making LLVM dependent on even MORE things that we do not
control. At least with the current system, we get to define the
interfaces, while putting in yet another tool, we are dependent on how
it does things.

I think the complexity of being dependent on large external libraries
(APR, boost, et al) is enough of a downside to not incorporate them.

> Downside:
> * Makes LLVM dependent on a third party library
> * Makes LLVM platform support dependent on
> * Error handling in APR is somewhat strange and it could be quite
> difficult for us to continue to meet the "throw std::string"
> approach we have today.

I vote against making LLVM dependent on even MORE things that we do not
control.

Okay. Vote registered. :slight_smile:

At least with the current system, we get to define the
interfaces, while putting in yet another tool, we are dependent on how
it does things.

Just to be very clear, we would still control the *interface* just not
the implementation. The lib/System headers would continue to exist, we
just wouldn't have separate implementations of the functionality for
each platform. Instead we'd use whatever small bits of APR were
applicable to LLVM. No APR header file would be visible outside the
implementation of lib/System.

I think the complexity of being dependent on large external libraries
(APR, boost, et al) is enough of a downside to not incorporate them.

Okay.

I think that optionally supporting a an APR interface would not be a bad
idea of obscure platforms, but LLVM should not depend on it. I think that
all of the major platforms should be natively supported by lib/System.

-Chris

I should also add that (I think) we should only use software that can be redistributed freely by us in any form (so that LLVM continues to work "out-of-the-box"), and which has a license roughly as liberal as LLVM does. In particular, we can't include any GPL software directly within the LLVM distribution. I looked through the Apache license and I think it's ok on these two issues:
  Apache License, Version 2.0

--Vikram
http://www.cs.uiuc.edu/~vadve
http://llvm.cs.uiuc.edu/

Hi everyone,

I think that all of the major platforms should be natively supported by
lib/System.

They would, through the APR.

Its not a major issue to me (and I'm a nobody here anyway), but I don't
understand why you would insist on this software independence *now*, when
LLVM is at a very early, and still largely unknown, stage.

Note: I don't know anything about APR, so I'm not specifically endorsing it,
just responding to the general idea that LLVM should be kept independent of
everything.

Note about Note: I would prefer you try to cover your dependencies using other
C++ libs first (for no particular reason other than I'm biased towards C++),
for example, boost probably has some things that we could use instead of APR
(boost-filesystem, boost-signals, boost-threads), but of course, APR may have
things you can't find anywhere else. In any event, what follows isn't
specific to any particular lib.

Use [insert-lib-here] to get up and running quickly on multiple platforms, and
if the dependence issue becomes a problem later when LLVM becomes more
popular, then come back and replace the [insert-lib-here]'s code. The point
is to get LLVM up and running *now*, and get it out in the world so it'll
start gathering mindshare, and more developers.

Looking at Ocaml as an example, which is *roughly* equivalent to what LLVM is
doing (bytecode compiler, native code compiler, virtual machine), except they
built it for a specific language, they have numerous dependencies on other
libs (curses, dbm, tcl, tk, etc).

Python is also similar, and its base has dependencies on 6 other libs besides
glibc, and that doesn't even count the add-on python modules (and what they
depend on).

If you want LLVM to become widespread and useful, you *will* end up either
depending on other libraries or spending a lot of time reinventing the wheel.
Your choice. :slight_smile:

FWIW,
Ed

Dear All,

Time to add my two cents:

I think incorporating something like APR into the LLVM tree is fine, given that it works, its licensing doesn't interefere with our licensing (and doesn't give me a headache), and we can merge it into the LLVM source base relatively seamlessly (i.e. users don't need to install it before building LLVM and APR plays nice with our build system).

I think building our own lib/System is going to be a bit of a time sink, especially with our limited access to other platforms. And adding third party libraries is okay as long as the user doesn't have to install extra stuff to use LLVM.

The licensing, I think, will be okay. The remainder of the problem lies with how well APR works and how well it will integrate with our build system. For that, I think we'll simply have to try it out and see if it works.

Are there any other libraries available that will do the things we need to do? It strikes me that we haven't enumerated what we need and what our options are.

If we go ahead and do incorporate APR, I would recommend the following:

a) Keep APR as a separate library and write lib/System as a wrapper around it.

b) Maintain a vendor branch for APR so that changes from the Apache Foundation are more easily merged into the tree (the CVS docs describe how to do this in the "Tracking Third Party Sources" section).

-- John T.

John,

If we were to do this, I don't think that adding it to the LLVM source
base is the right way to go. We would simply use "configure" to find the
library and header files. The moment we put APR into our source base, it
would be out of date. Keeping it up to date would not be fun for anyone
and there's no reason for us to do that. Furthermore, this approach
completely avoids any licensing issues. We wouldn't distribute APR, just
use it (even though Apache's license is relatively tame).

As for other libraries, there is boost (which we've already excised),
and ACE (which is huge and heavy weight). APR is the rising star in this
area.

I think Chris had the right idea: make APR "one" of the possible
implementations. That is, make it possible for the user to configure
LLVM so that it thinks the operating system its building for is "APR".
All we have to do is create an APR directory in lib/System and the
necessary functions in configure.ac to allow it to be specified as the
host operating system. I think I might do this regardless of what the
decision on this issue is because it would at least give new platforms a
shot at having LLVM work.

Reid.

Reid Spencer wrote:

John,

If we were to do this, I don't think that adding it to the LLVM source
base is the right way to go. We would simply use "configure" to find the
library and header files. The moment we put APR into our source base, it
would be out of date. Keeping it up to date would not be fun for anyone
and there's no reason for us to do that. Furthermore, this approach
completely avoids any licensing issues. We wouldn't distribute APR, just
use it (even though Apache's license is relatively tame).

As for other libraries, there is boost (which we've already excised),
and ACE (which is huge and heavy weight). APR is the rising star in this
area.

Okay.

We excised boost because we were only using a small subset of it that we could have written ourselves. If boost as a whole does what we need, I think it could become an option again (although, IIRC, boost does not have uniform copyrights like Apache does, we was giving me a headache, so that might be an issue).

Were there any other reasons to not consider boost? I'm not suggesting we use it; I'm just curious.

I think Chris had the right idea: make APR "one" of the possible
implementations. That is, make it possible for the user to configure
LLVM so that it thinks the operating system its building for is "APR".
All we have to do is create an APR directory in lib/System and the
necessary functions in configure.ac to allow it to be specified as the
host operating system. I think I might do this regardless of what the
decision on this issue is because it would at least give new platforms a
shot at having LLVM work.

I think this approach would be good for transitioning over to APR.

However, I don't really like the idea of having two facilities in LLVM to do the same thing (namely, platform abstraction). If APR lives up to its goal, then we should eventually be able to use it as *the* platform abstraction library, in which case, I think it should go into the source tree so that people don't have to install it in order to install LLVM.

I think trying out APR as you and Chris have suggested above would be a good way to see whether or not it will really work without committing completely to it. When we know it will work, we can discuss whether it should go into the source tree or not. Cross that bridge when we get to it sort of a thing.

-- John T.

I agree with what you're saying, but there are a couple of different
issues here. What we really want is for the LLVM *core* to be as
independent of other libraries as possible. It should be possible to just
untar, build and install, without having to worry about tons of obscure
dependencies. There are multiple ways to do this (e.g. include the
dependencies into the LLVM distro), we just want to be smart about how we
do it.

Note that projects that USE llvm will probably have dependencies of their
own. For example, the llvm-java project will probably have a lot of the
dependencies that you described for other language runtimes. Specifically
we just want the core of LLVM to be as simple as possible.

This is just MHO of course. :slight_smile:

-Chris

Were there any other reasons to not consider boost? I'm not suggesting
we use it; I'm just curious.

The main reason we got rid of boost is it's configuration system which
looks at particular compilers and versions. This was a pain to maintain
(it needed to be hacked every time a new gcc came out). Also, boost
provides a ton of stuff we don't need, and does not provide a ton of stuff
we do need. *shrug*

However, I don't really like the idea of having two facilities in LLVM
to do the same thing (namely, platform abstraction). If APR lives up to
its goal, then we should eventually be able to use it as *the* platform
abstraction library, in which case, I think it should go into the source
tree so that people don't have to install it in order to install LLVM.

Again, as mentioned, APR does not provide everything we need, so I don't
know if there is a way around this.

-Chris

John Criswell wrote:

Okay.

We excised boost because we were only using a small subset of it that we
could have written ourselves. If boost as a whole does what we need, I
think it could become an option again (although, IIRC, boost does not
have uniform copyrights like Apache does, we was giving me a headache,
so that might be an issue).

Regarding copyrights:
1. All of them fulfill Boost copyright requirements, which means they all as
as free as possible.
2. An effort is going on to move all Boost to a unified license
(http://boost.org/LICENSE_1_0.txt) and a lot of process is made already.

Just FYI.

- Volodya

Reid,

Adding APR as one possible implementation of lib/System makes sense, and is what I originally suggested when I brought up the question of using APR. In particular, I agree that we want to keep APR or any other similar layer encapsulated behind lib/System.

--Vikram
http://www.cs.uiuc.edu/~vadve
http://llvm.cs.uiuc.edu/

Reid,

Adding APR as one possible implementation of lib/System makes
sense,and is what I originally suggested when I brought up the
question ofusing APR. In particular, I agree that we want to keep APR
or anyother similar layer encapsulated behind lib/System.

Yes.

What I might do is send the APR email list a "wish list" of things we'd
like to see. If/when they get implemented on enough platforms, I could
do the port. Doing it now is just silly because there's such little
support for things we need.

Reid.

I reallize I am responding to this thread a bit late, but here goes.

Snip snip snip, cut cut cut
As for other libraries, there is boost (which we've already excised),
and ACE (which is huge and heavy weight). APR is the rising star in this
area.

As someone who has had to work on and with software written against ACE
(the TAO codebase in fact) may I strongly suggest that you give no
further time to this idea. Really, if for no other reason than the time
it takes gdb to load the debug synbols for ACE can be on the order of a
couple minutes. To say it is heavy weight is an understatement. It
insists on working on very non-standards compliant C++ compiles (i.e.
ones with little support for templates or stl), thus reimplements many
things in the standard library in a really C-type-unsafe way.

Just my 1/2 cent on ACE.

As for APR, when I had glanced at it briefly, it (of course) seemed
rather more focused on providing the things you need for sane cross
platform networking code. It wasn't clear to me that this really
translated into providing everything we need nor provided useful
abstractions for what we did. And do we really want to write a wrapper
to wrap another wrapper?

Andrew

Agreed on all points. I've used ACE on a commercial product and its a behemoth. Debugging as you suggested is a nightmare.

APR is focused on networking support (and it does a pretty good job there) but that's to be expected since its principal use is in Apache HTTPD. For lots of things that we need, APR just has no support which is why it won't get implemented in lib/System for a while (until it can do things that we need it to do).

And that's a "wrap".

Reid.

Andrew Lenharth wrote:

mån 2004-09-13 klockan 06:45 +0000 skrev en okänd avsändare:

I should also add that (I think) we should only use software that can
be redistributed freely by us in any form (so that LLVM continues to
work "out-of-the-box"), and which has a license roughly as liberal as
LLVM does. In particular, we can't include any GPL software directly
within the LLVM distribution. I looked through the Apache license and
I think it's ok on these two issues:
  Apache License, Version 2.0

According to Various Licenses and Comments about Them - GNU Project - Free Software Foundation the Apache
Software License, version 2.0 is incompatible with the GPL. If LLVM
would require the APR, then this could mean that GPL software would not
be allowed to use LLVM, or perhaps only use it in certain ways (for
example exec:ing binaries but not linking to libraries).

/abo