Multiple directories in a single library

Hi all,

I've previously posted this patch on llvm-commits, but due to a lack of
replies, and the fact that this patch is more something to discuss than
something to apply, I'm posting this again here.

While working on my own backend, I found that things got really messy real
quickly, partly caused by the fact that all .cpp files must be in the same
directory (lib/Target/TargetName). Simply putting code in different
directories and using DIRS in the Makefile doesn't cut it, since that produces
different libraries for each directory, which don't get linked in by the
programs that need them.

There is some code in llvm-config which fixes the library problem for the
TargetNameAsmPrinter library, but that isn't really the clean way IMHO. Is
this also meant for cleanup, or is there another reason why the AsmPrinter
should be in a seperate library?

Anyway, I thought it would be good if one could simply compile .cpp files in
subdirectories as if they are in the current directory, without a makefile in
the subdirectory. These cpp files would end up in the library as defined by
the parent directory's Makefile.

Is there any interest in this, or am I the only one who would like to have
something like this?

I've attached a small patch which facilitates this. It allows you to define
the EXTRA_SOURCES variable to explicitely add extra source files (one could
also explicitly define SOURCES, but this way you don't loose the default
all-sources-in-the-current-dir feature). For example, I would add:
  EXTRA_SOURCES = $(wildcard $(PROJ_SRC_DIR)/SubDir/*.cpp)

to get all the cpp files in SubDir compiled as well.

It would be more elegant to specify EXTRA_DIRS instead (or perhaps a better
name) as a list of directories, and let Makefile.rules handle the wildcarding
here. However, I've not found a way to write
.for DIR in $(EXTRA_DIRS)
  SOURCES += $(patsubst $(PROJ_SRC_DIR)/%, %, wildcard $(PROJ_SRC_DIR)/$DIR/*.cpp))
.endfor

in the Makefile (the above appears to be valid in NetBSD make, any suggestions
on how to do this in GNU make?) Doing this in shell seems tricky, because then
the wildcarding must be done in shell as well AFAICS.

The second change in the patch makes sure that the proper directory is created
below $(ObjDir). My first attempt was to make the build rules depend on
$(ObjDir)/SubDir/.dir but this didn't quite work out (I didn't manage to get
just "SubDir" from the filename in a clean way, and make threw away the .dir
files afterwards, since they were intermediate targets).

The alternative in the patch is to just directly mkdir in the build target,
which works.

Gr.

Matthijs

extra-sources.diff (1.34 KB)

For what it’s worth, I have lib/Target compiling into one library. It works for building dynamic libraries, but I’m having some trouble with getting static archive builds to load targets. All subdirectories are included, identical base names should be handled gracefully, and compilation in a subdirectory will have all parent (sub)directories in their header search path.

A few examples of how I did it:

<http://www.bitbucket.org/danchr/llvm/src/253dfa2dabed/lib/Target/Makefile#cl-10>
<http://www.bitbucket.org/danchr/llvm/src/253dfa2dabed/Makefile.config.in#cl-82>
<http://www.bitbucket.org/danchr/llvm/src/253dfa2dabed/Makefile.rules#cl-532>
<http://www.bitbucket.org/danchr/llvm/src/253dfa2dabed/Makefile.rules#cl-589>
<http://www.bitbucket.org/danchr/llvm/src/253dfa2dabed/Makefile.rules#cl-607>

(The server seems rather slow when querying individual files; sorry…)

Hello, Matthijs

There is some code in llvm-config which fixes the library problem for the
TargetNameAsmPrinter library, but that isn't really the clean way IMHO. Is
this also meant for cleanup, or is there another reason why the AsmPrinter
should be in a seperate library?

The main reason of such split was codesize concerns for JIT users:
they don't need asmprinting at all. The ugly include paths hacks were
due to current lack of possibility to "export private headers".

Anyway, I thought it would be good if one could simply compile .cpp files in
subdirectories as if they are in the current directory, without a makefile in
the subdirectory. These cpp files would end up in the library as defined by
the parent directory's Makefile.

Then, this again "merges" asmprinters into whole target library, right?

Hi Anton,

The main reason of such split was codesize concerns for JIT users:
they don't need asmprinting at all. The ugly include paths hacks were
due to current lack of possibility to "export private headers".

Ah, I suspected as much.

> Anyway, I thought it would be good if one could simply compile .cpp files in
> subdirectories as if they are in the current directory, without a makefile in
> the subdirectory. These cpp files would end up in the library as defined by
> the parent directory's Makefile.
Then, this again "merges" asmprinters into whole target library, right?

If you would use the EXTRA_SOURCES for the AsmPrinter dir, then yes. But there
is nothing to stop you from keeping AsmPrinter in DIRS with its own makefile,
and putting other dirs in EXTRA_SOURCES.

Gr.

Matthijs