One of the things that I don't understand well about LLVM is what code
is in what object files or library archives. It would be very useful if
there was a map of the dependencies between the files (e.g. if you link
X.o you need Y.a and Z.o). Trying to figure out the link lines by trial
and error is a bit frustrating.
To assist myself with understanding this, I've started to write a
document (ObjectFiles.html, attached) that will (eventually) explain all
this. Right now its pretty much just a list of the various files
produced by building LLVM and a cursory statement of what the file
contains. The most significant need in the document is for a section
on the dependencies between the various files. Some more "rules of
thumb" about how to link against LLVM would be good too.
Could someone please add this file to the LLVM docs directory so I can
submit patches against it?
I'll continue to work on this as my understanding improves. HOWEVER,
feel free to embellish. This isn't a document I need to _write_, its a
document I need to _read_!
One of the things that I don't understand well about LLVM is what code
is in what object files or library archives. It would be very useful if
there was a map of the dependencies between the files (e.g. if you link
X.o you need Y.a and Z.o). Trying to figure out the link lines by trial
and error is a bit frustrating.
No kidding. We were just talking about how it would be nice to have some
sort of automatic library dependency manager (perhaps hacked together with
nm), but came to the conclusion that it would be more trouble than it's
worth.
To assist myself with understanding this, I've started to write a
document (ObjectFiles.html, attached) that will (eventually) explain all
this. Right now its pretty much just a list of the various files
produced by building LLVM and a cursory statement of what the file
contains. The most significant need in the document is for a section
on the dependencies between the various files. Some more "rules of
thumb" about how to link against LLVM would be good too.
Sounds good.
Could someone please add this file to the LLVM docs directory so I can
submit patches against it?
Done.
I'll continue to work on this as my understanding improves. HOWEVER,
feel free to embellish. This isn't a document I need to _write_, its a
document I need to _read_!
Sounds good. I'm concerned that this document might be hard to keep
up-to-date, but it's clearly something that is needed.
libinstrument.a -> instrumentation (e.g. profiling) transformations
libtransformutils.a -> common code shared by various xforms
libregalloc.a -> sparc register allocator
sched.o -> sparc instruction scheduler
select.o -> sparc instruction selector support
libevar.o -> live variable analysis for the sparc
selectiondag.o -> start of aggressive instruction selector for DAGs
profpaths.o -> path profiling instrumentation
Basically, libraries are built in two forms: .a files and .o files. .o
files are built/used when client programs want to link in _everything_ in
a library. .a files are used when a client program wants to use some of
the stuff in a library, based on what symbols are referenced.
The difference is important for things like 'opt' which wants to link in
_ALL_ optimizations no matter what, and 'gccas' which only wants to link
optimizations in that are referenced by the gccas tool explictly.
For libraries that .a files don't make sense, we don't provide them. It's
impossible to link in half an x86 backend for example.
Overall, this looks really useful, especially when the dependency
information is built. Perhaps this should eventually be integrated into
the programmers manual though?
Perhaps, but its a good idea and would eliminate the need to manually
update the document. I could probably hack something together with perl
+ nm pretty quickly. Wouldn't be fast, wouldn't be pretty but at least
it would figure out the maze of dependencies. It could even output in
html to be included in the document. I'll see what I can hack together.
Don't hold your breath though, I'm in the middle of moving to WA state
on Monday!
> > Could someone please add this file to the LLVM docs directory so I can
> > submit patches against it?
BTW, here's some hints for it:
Thanks Chris, I'll add them.
Basically, libraries are built in two forms: .a files and .o files. .o
files are built/used when client programs want to link in _everything_ in
a library. .a files are used when a client program wants to use some of
the stuff in a library, based on what symbols are referenced.
The difference is important for things like 'opt' which wants to link in
_ALL_ optimizations no matter what, and 'gccas' which only wants to link
optimizations in that are referenced by the gccas tool explictly.
For libraries that .a files don't make sense, we don't provide them. It's
impossible to link in half an x86 backend for example.
While I understand the motivations you describe above, something about
this bothers me. As a tool provider the LLVM developers are trying to
provide small/fast/quick linking tools. The above approach helps with
some of that. However, I'm an LLVM user and a big part of using LLVM is
linking my code with LLVM code. When the LLVM developers opt to make a
.o "library", they're essentially forcing users to take all or nothing.
As you mentioned in the case of x86 backend, this may make a lot of
sense. On the other hand, it presumes the developers of LLVM know how
the users want to use LLVM! Say there's a little utility function in
the x86 backend that I want to use but I'm not generating any x86 code?
Overall, this looks really useful, especially when the dependency
information is built. Perhaps this should eventually be integrated into
the programmers manual though?
Perhaps, but with hypertext, what does "integrated" really mean?
While I understand the motivations you describe above, something about
this bothers me. As a tool provider the LLVM developers are trying to
provide small/fast/quick linking tools. The above approach helps with
some of that. However, I'm an LLVM user and a big part of using LLVM is
linking my code with LLVM code. When the LLVM developers opt to make a
.o "library", they're essentially forcing users to take all or nothing.
Exactly.
As you mentioned in the case of x86 backend, this may make a lot of
sense. On the other hand, it presumes the developers of LLVM know how
the users want to use LLVM! Say there's a little utility function in
the x86 backend that I want to use but I'm not generating any x86 code?
If you're not generating X86 code, don't use the X86 library. The
idea is that LLVM is broken up into libraries that are already probably
already too fine-grained. If you don't need something, just don't link to
it...
> Overall, this looks really useful, especially when the dependency
> information is built. Perhaps this should eventually be integrated into
> the programmers manual though?
Perhaps, but with hypertext, what does "integrated" really mean?
Well, in the long term we want to switch everything over to docbook, at
which point it should just be a matter of doing the equiv of "#include
ObjectFiles.html", and we suddenly have a new section in the book...
> As you mentioned in the case of x86 backend, this may make a lot of
> sense. On the other hand, it presumes the developers of LLVM know how
> the users want to use LLVM! Say there's a little utility function in
> the x86 backend that I want to use but I'm not generating any x86 code?
If you're not generating X86 code, don't use the X86 library. The
idea is that LLVM is broken up into libraries that are already probably
already too fine-grained. If you don't need something, just don't link to
it...
This works as long as the libraries are fine-grained _enough_. They
probably are so I'm not going to make an issue out of this because there
are bigger fish to fry.
Well, in the long term we want to switch everything over to docbook, at
which point it should just be a matter of doing the equiv of "#include
ObjectFiles.html", and we suddenly have a new section in the book...
DocBook is good. I'd rather use that than HTML. There's got to be an
HTML to DocBook translator around *somewhere*. Is there a Bugzilla for
this work? Need me to work on this or has it already started?
This works as long as the libraries are fine-grained _enough_. They
probably are so I'm not going to make an issue out of this because there
are bigger fish to fry.
If not, they can always be split up later.
> Well, in the long term we want to switch everything over to docbook, at
> which point it should just be a matter of doing the equiv of "#include
> ObjectFiles.html", and we suddenly have a new section in the book...
DocBook is good. I'd rather use that than HTML. There's got to be an
HTML to DocBook translator around *somewhere*. Is there a Bugzilla for
this work? Need me to work on this or has it already started?
The first step (PR120) is to convert all of our documentation to HTML4.01
strict, which Misha has been working on. I think this is getting close to
done, but is stalled for the moment as Misha is working on other things.
Once that's done, PR121 is for conversion to docbook, which we hope to
automate as much as possible.
If you'd like to help (which we would certainly appreciate), grab one of
the documents in the docs/ directory that is not marked 4.01 strict, and
hack on it until it is.
Okay, 4.01 Strict I grok and the w3c has automated compliance checkers
for it. So, I'll start working on this after my computer gets unpacked
on Tuesday.
