Proposed Makefile Changes

Hi,

In buildling XPS using LLVM's makefile system, I'm finding that there's
a few things lacking in our support for LLVM-based projects. The items
below should help but may require changes to project makefiles. I
thought I'd check before just going and doing it.

     1. Various autoconf generated variables (e.g. abs_top_srcdir) are
        set in the makefiles but not used. They conflict with a user's
        project that might want to use them. I propose removing them or
        using names prefixed with llvm_ to ensure their use is only for
        llvm
     2. BUILD_SRC_DIR and BUILD_OBJ_DIR are poorly named. This has been
        a source of confusion for me personally and probably for others.
        They should really be PROJ_SRC_DIR and PROJ_OBJ_DIR to indicate
        they pertain to the project's src/obj dir not LLVM's
     3. The LLVM makefiles require a Makefile.config in the project. I'm
        going to try to make that optional because some projects can
        just use LLVM's Makefile.config.
     4. There should be no "default" directives because it prevents the
        use of a directive in the user's Makefile to document what will
        be built. Currently, the only such case is that we build a
        "relinked" object file by default for each library unless
        DONT_BUILD_RELINKED is specified. While this is common for LLVM,
        it isn't for projects which will want an archive or shared
        library. If a relinked library is needed, something link
        RELINKED_LIBRARY=1 should be specified so that it documents in
        the makefile what is getting built.
     5. Some of the names of the directives are a little confusing. We
        have BUILD_ARCHIVE to specify building an archive. But, we have
        SHARED_LIBRARY to specify building a shared library. I think
        these should all be named BUILD_<thing_to_build> so we'd have
        BUILD_ARCHIVE_LIBRARY and BUILD_SHARED_LIBRARY and
        BUILD_RELINKD_LIBRARY and BUILD_TOOL
     6. The directive names are concatenated in some cases. I'd like to
        see TOOL_NAME instead of TOOLNAME and LIBRARY_NAME instead of
        LIBRARYNAME. Its easier for *me* to read, but I'd like some
        feedback on this one as its as "gratuitous" change.
     7. The documentation in MakefileGuide.html is lacking a tutorial on
        how to use the LLVM Makefile system for projects (as opposed to
        LLVM itself). I think its time to add one. There are things in
        the configure.ac file that are needed to support the makefile
        system. Those need to be documented as well.

The end result should be that using the LLVM Makefile for things other
than LLVM itself gets a easier/clearer and better documented.

I'm signing up to do the necessary changes in llvm and llvm-test. I'll
also help anyone with the necessary changes in their project.

Please let me know if these changes will be a significant burden or they
are inadvisable for your project.

Thanks,

Reid.

In buildling XPS using LLVM's makefile system, I'm finding that there's
a few things lacking in our support for LLVM-based projects. The items
below should help but may require changes to project makefiles. I
thought I'd check before just going and doing it.

ok.

    1. Various autoconf generated variables (e.g. abs_top_srcdir) are
       set in the makefiles but not used. They conflict with a user's
       project that might want to use them. I propose removing them or
       using names prefixed with llvm_ to ensure their use is only for
       llvm

If these are really not used, I don't see any harm in removing them. You should check with people running complex external projects (like the reoptimizer) to find out if they are really dead though.

    2. BUILD_SRC_DIR and BUILD_OBJ_DIR are poorly named. This has been
       a source of confusion for me personally and probably for others.
       They should really be PROJ_SRC_DIR and PROJ_OBJ_DIR to indicate
       they pertain to the project's src/obj dir not LLVM's

I totally agree here.

    3. The LLVM makefiles require a Makefile.config in the project. I'm
       going to try to make that optional because some projects can
       just use LLVM's Makefile.config.

Ok. Does this mean you want to make the project configure script optional, or just a Makefile.config? What is the harm in leaving it? Does removing it actually gain anything?

    4. There should be no "default" directives because it prevents the
       use of a directive in the user's Makefile to document what will
       be built. Currently, the only such case is that we build a
       "relinked" object file by default for each library unless
       DONT_BUILD_RELINKED is specified. While this is common for LLVM,
       it isn't for projects which will want an archive or shared
       library. If a relinked library is needed, something link
       RELINKED_LIBRARY=1 should be specified so that it documents in
       the makefile what is getting built.

I think this is fine.

    5. Some of the names of the directives are a little confusing. We
       have BUILD_ARCHIVE to specify building an archive. But, we have
       SHARED_LIBRARY to specify building a shared library. I think
       these should all be named BUILD_<thing_to_build> so we'd have
       BUILD_ARCHIVE_LIBRARY and BUILD_SHARED_LIBRARY and
       BUILD_RELINKD_LIBRARY and BUILD_TOOL

I would also like to see this happen, but it's a big change. I suspect that people running projects would prefer to not change all of these.

    6. The directive names are concatenated in some cases. I'd like to
       see TOOL_NAME instead of TOOLNAME and LIBRARY_NAME instead of
       LIBRARYNAME. Its easier for *me* to read, but I'd like some
       feedback on this one as its as "gratuitous" change.

Likewise.

    7. The documentation in MakefileGuide.html is lacking a tutorial on
       how to use the LLVM Makefile system for projects (as opposed to
       LLVM itself). I think its time to add one. There are things in
       the configure.ac file that are needed to support the makefile
       system. Those need to be documented as well.

There is some information here:
http://llvm.cs.uiuc.edu/docs/Projects.html

But it's not really a tutorial. A tutorial would be great to have, but we should figure out the relationship between these two docs.

The end result should be that using the LLVM Makefile for things other
than LLVM itself gets a easier/clearer and better documented.

yaay. :slight_smile:

I'm signing up to do the necessary changes in llvm and llvm-test. I'll
also help anyone with the necessary changes in their project.

If you make these changes, please make sure to send instructions for exactly what needs to be done to update projects. These kinds of things can lead to all sorts of obscure problems for those who aren't experts in the build system...

-Chris

> 3. The LLVM makefiles require a Makefile.config in the project. I'm
> going to try to make that optional because some projects can
> just use LLVM's Makefile.config.

Ok. Does this mean you want to make the project configure script
optional, or just a Makefile.config? What is the harm in leaving it?
Does removing it actually gain anything?

The problem was just a bug in the makefiles. It had a rule that made
project's Makefile.config depend on LLVM's Makefile.config.in which is
obviously broken because it completely thwarted the project's
Makefile.config.

I thought that for some really small projects, there wouldn't be any
need for a Makefile.config.in or configure scripts since LLVM provides
portability and everything that's needed. It might encourage some
additional users if the cost to getting started was reduced. Basically
they can hand code a Makefile.common and start compiling. If the
project has a Makefile.config.in then it will assume full autoconf
support (reconfigure target etc.) for the project, otherwise not. Make
sense?

> 5. Some of the names of the directives are a little confusing. We
> have BUILD_ARCHIVE to specify building an archive. But, we have
> SHARED_LIBRARY to specify building a shared library. I think
> these should all be named BUILD_<thing_to_build> so we'd have
> BUILD_ARCHIVE_LIBRARY and BUILD_SHARED_LIBRARY and
> BUILD_RELINKD_LIBRARY and BUILD_TOOL

I would also like to see this happen, but it's a big change. I suspect
that people running projects would prefer to not change all of these.

Yes, I know. I think what I'll do is support both names temporarily with
a make "warning" to caution the user to change the names, if the old
names are used. That way it doesn't break but its just annoying enough
that it might provide the impetus for project owners to update the
variable names.

> 7. The documentation in MakefileGuide.html is lacking a tutorial on
> how to use the LLVM Makefile system for projects (as opposed to
> LLVM itself). I think its time to add one. There are things in
> the configure.ac file that are needed to support the makefile
> system. Those need to be documented as well.

There is some information here:
http://llvm.cs.uiuc.edu/docs/Projects.html

But it's not really a tutorial. A tutorial would be great to have, but we
should figure out the relationship between these two docs.

Okay, forgot about Projects.html. I think I will leave
MakefileGuide.html as the "reference" manual and place the more
descriptive and project-focused tutorial into Projects.html.

Make sense?

If you make these changes, please make sure to send instructions for
exactly what needs to be done to update projects. These kinds of things
can lead to all sorts of obscure problems for those who aren't experts in
the build system...

I will. And I'll try to be as backward-compatible as possible.

Reid

The proposed makefile changes have been committed. If you are working
from CVS head and you use the LLVM Makefile System in your own project,
please make a note of the following:

     1. If your makefiles use any BUILD_* variables, they now need to be
        prefixed with PROJ_ instead of BUILD_. For example,
        BUILD_SRC_ROOT is now PROJ_SRC_ROOT.
     2. There are additional requirements for projects. Please review
        this file:
        http://illuvium.net/src/projects/sample/Makefile.common.in

Okay .. that went off premature .. I keep thinking Ctrl-Enter gets me a
newline in a list but it sends the message. Sorry. I'll repost the final
thing in a few minutes.

The proposed makefile changes have been committed. If you are working
from CVS head and you use the LLVM Makefile System in your own project,
please make a note of the following:

1. If your makefiles use any BUILD_* variables, they now need to be
    prefixed with PROJ_ instead of BUILD_. For example,
    BUILD_SRC_ROOT is now PROJ_SRC_ROOT.

2. There are additional requirements for projects. Please review
    this file:
       http://illuvium.net/src/projects/sample/Makefile.common.in
    as it contains the complete list of variables you need to set and
    provides some documentation on them. Note that there are three
    new project related variables you need to set: PROJECT_NAME,
    PROJ_VERSION, and PROJ_INSTALL_DIR. These variables make it
    possible for the "install", "dist", and "dist-check" targets to work
    for your project too, not just LLVM.

3. If it isn't clear how to set up your project, please consult the
    documentation here: http://llvm.cs.uiuc.edu/docs/Projects.html

If you have any questions/issues with these changes, please let me know.
I'd be happy to help and resolve any issues.

Please note that the llvm-java project has already been converted.

Reid.

The llvm.cs.uiuc.edu site does not seem to be updating at the moment so
it contains the old instructions. If you're looking for the new
instructions on the Projects.html page, you can view them here:
http://illuvium.net/docs/Projects.html

Reid.

The llvm.cs.uiuc.edu site does not seem to be updating at the moment so
it contains the old instructions. If you're looking for the new
instructions on the Projects.html page, you can view them here:
http://illuvium.net/docs/Projects.html

Fixed.

-Chris