Optional Target Builds

Project Infrastructure LLVM Dev List Archives
1

With Misha’s changes requiring a total rebuild, I’ve been reminded how long the targets take to build. Now that there are more of them and in most cases I really don’t care about n-1 of them, I’m wondering if its time to have configure allow optional compilation for them.

I would like to suggest:

  1. The default (build all targets) remains the same
  2. We add to configure --disable-target-x86 to disable building the x86 target. Similarly for all other targets
  3. We add to configure --disable-foreign-targets to disable all but the current host’s target type.

This will require a little bit of configure magic and some changes to the makefiles.

Anyone have a good reason not to do this?

Reid

2

Sounds great to me. I would suggest --disable-cross-targets instead of --disable-foreign-targets though.

-Chris

3 
> With Misha's changes requiring a total rebuild, I've been reminded how
> long the targets take to build. Now that there are more of them and in
> most cases I really don't care about n-1 of them, I'm wondering if its
> time to have configure allow optional compilation for them.
>
> I would like to suggest:
>
> 1. The default (build all targets) remains the same
> 2. We add to configure --disable-target-x86 to disable building the x86
> target. Similarly for all other targets
> 3. We add to configure --disable-foreign-targets to disable all but the
> current host's target type.
>
> This will require a little bit of configure magic and some changes to
> the makefiles.

Sounds great to me.  I would suggest --disable-cross-targets instead of 
--disable-foreign-targets though.

Actually, the options have to be expressed as “enable-something” so I’ve opted for “–enable-target-this” to enable just the $host target. if you specify --disable-target-this (the default) then you get whatever platforms you specify with --enable-target-{targetname}

Does that make sense or do you have a better idea?

4

-enable-targets=x86,alpha,sparcv9
-link-targets=alpha,host

Valid options for both are:
the names of the targets
host
all

where all is the default for both items

I have in my tree the patch to partially support this (well the link
half of it). Right now I am controlling linking with a variable in the
toplevel makefile until a config patch like this goes in.

Andrew Lenharth
http://www.lenharth.org/~andrewl/

5

What I would prefer (for what that's worth) is that by default,
only the host target is enabled. That will speed up my builds
a bit and keep sizes a bit smaller. If I then had an option so
I could do something like '--enable-arch=ppc,x86_64,ia64', I
could build whatever cross-targets I wanted for a particular
package.

6

I think this makes the most sense. I assume that enabled-but-not-linked targets are built into shared objects?

My one concern about this change is that it makes it more likely for people to break the build on hosts not equal to their own. However, as long as the nightly testers due the full build, I dpm
t think this is a big problem.

-Chris

7

Does that make sense or do you have a better idea?

-enable-targets=x86,alpha,sparcv9
-link-targets=alpha,host

Valid options for both are:
the names of the targets
host
all

where all is the default for both items

I have in my tree the patch to partially support this (well the link
half of it). Right now I am controlling linking with a variable in the
toplevel makefile until a config patch like this goes in.

I think this makes the most sense. I assume that enabled-but-not-linked targets are built into shared objects?

This makes sense to to me too, except I agree with a couple of the others that it would be most convenient to build only for the host machine by default. That is nearly always needed and most likely to minimize redundant compile/link time.

8

To me, getting testing is more important than user convenience in this case. In particular, people who build on slow machines often can just use the configure option to just build and link in the target matching the host. If we default to building everything then new users (or people who want to test everything on a regular basis) will build everything by default, but people on slow machines can choose to disable certain chunks.

-Chris

9

Would passing one option, "--enable-arch=host", be ok?

-Chris

10

If what you mean is that "--enable-arch=host" would build only the
host target, that could work. "--enable-arch=host-only" or something
_might_ be clearer. So let me see if I can paraphrase: the proposal
is to add a new configure option to the build, "--enable-arch" (or
"--enable-target" -- either works for me). One can provide comma
separated parameters to "--enable-arch" as follows:

   -- "all": the default, which builds all targets, regardless of
      the host for the build
   -- "host" (or "host-only"): build only the code generators for
      the current build host
   -- "c", "x86", "ppc", ... : build only the code generators for the
      named targets (hmm, maybe '--enable-target' is better..).

If that's what you meant, it sounds good to me... but I reckon
Reid gets to have a say, too :).

11 
-enable-targets=x86,alpha,sparcv9
-link-targets=alpha,host

Valid options for both are:
the names of the targets
host
all

As others have agreed, this is a much better approach than the one I was thinking of. Its harder to parse in the configure script, but I can probably find a way to do it.


where all is the default for both items

There has been some debate about the default value. I tend to agree with Chris on this. The default should be “all” so that everything gets tested by default. More sophisticated users can limit the targets that are built by merely typing -enable-targets=host on the configure line; not a big deal in my books. One other problem with making the default build only one or some of the targets is that it will (currently) cause the dejagnu tests to fail because llc won’t know about certain targets that are in the test suite.

Rei.d

12

Yeah, that’s pretty much exactly it, with one exception. I was planning to always build the CBackend and Skeleton targets. Both are pretty fast to build and it gives LLVM some kind of target to fall back on. Given that, we might also want to have an option for “none” so that no processor targets are built, just the CBackend and Skeleton.

Thoughts?

Reid

13

Cosmetically, I like "host-only" instead of "host". I think that the skeleton target should not be special: it should be treated just like X86 or any other target (i.e. it is disabled when host-only is used). I agree that cbe should always be enabled.

w.r.t. dejagnu tests, don't worry about it. This is just a matter of enhancing the dj framework to be a bit smarter. We can do this as a second step.

-Chris

14

There has been some debate about the default value. I tend to agree
with Chris on this. The default should be "all" so that everything
gets tested by default. More sophisticated users can limit the targets
that are built by merely typing -enable-targets=host on the configure
line; not a big deal in my books.

I would claim that from a user's perspective, default should be a
compiler that generates code for the host platform, and nothing else.
As we gather more and more backends, the "default" compilation will take
longer and longer.

A developer who wishes to make sure none of his changes break other
targets and the nightly tester should run with `--enable-targets=all',
but it doesn't make sense (to me) to make it the default.

One other problem with making the default build only one or some of
the targets is that it will (currently) cause the dejagnu tests to
fail because llc won't know about certain targets that are in the test
suite.

That will have to be changed regardless of what setting is chosen for
the --enable-targets default, because if a user compiles only a small
set of targets, it doesn't make sense for the test suite to report a lot
of errors for targets the user isn't even remotely interested in.

15

That sounds good. Like you said, it guarantees at least one
usable target. Go for it :).

16

Okay, we need to get this resolved. So far we have two camps: (a) those
that want the default to be "all" and (b) those that want the default to
be "host-only". This is easy enough to change in the configure script;
we just need to make a decision. So far we have:

(a) - Reid, Chris, Andrew Lenharth
(b) - Misha, Vikram, Al Stone

Since we're split down the middle (including the Oversight committee),
I'm not sure which way to go.

Anyone want to change their vote? Any other voters? Any other way to
resolve this?

Reid

17

Okay, we need to get this resolved. So far we have two camps: (a) those
that want the default to be "all" and (b) those that want the default to
be "host-only". This is easy enough to change in the configure script;
we just need to make a decision. So far we have:

(a) - Reid, Chris, Andrew Lenharth

I vote for A because at this stage LLVM is primarily used by developers and with all the changes made.. its good to have it set by default to compile everything, and run tests for all targets. Not everyone really checks the nightly-tester (ie. how many notice the sparc test failures or even care?). Now if someone is working on a specific backend where changes do not impact other targets, they can easily configure to only compile that one target.

I think this subject should be revisited when llvm becomes more mainstream.

-Tanya

18

Okay, Tanya and Markus tipped the decision over to option (a), build all
targets. However, I think Tanya hit it right on the money: this needs to
be revisited when the "activity community" is more users than
developers.

The feature is now implemented in the CVS head. If you do nothing you
shouldn't experience any changes as the default is the same as before
the feature was implemented.

To use the feature, the option to configure is --enable-targets={value}.
Valid values are:
  all: build all targets
  host-only: build only the target for the host you're building on
  list: comma separated list with specific targets selected from
        alpha,ia64,powerpc,skeleton,sparc,x86,x86_64

Note that for now, x86_64 and x86 are the same. They both enable
building the lib/Target/X86 directory.

I would appreciate it if, on the next reconfigure, people using PowerPC,
Sparc, IA64 and Alpha would test this feature out. In particular, please
validate that:

The default is "all", "host-only" gets your platform correct, and some
combination of specific targets works correctly.

Thanks for all the feedback and lively debate on this.

Reid

19

Let me try to clarify why I feel the way I do. For this discussion, I use the term "end-user" to be someone who just wants to USE a compiler, and doesn't want to hack on the compiler.

To me, the current LLVM distro isn't something that an end-user would generally want to use directly. In particular, the LLVM CVS repo is currently biased towards supporting developers more than end-users. For end-users, I think it would be most useful to build all of the targets, but only link the host target into llc (the others remain as .so files). That way the common case of non-cross-compiling will be fast (llc has a smaller disk footprint), but cross compilation can still work.

Since end users don't build the tree often (e.g. only once), they don't really care THAT much about LLVM build time. More commonly, a packager will do this work for them, producing a set of debs, gentoo packages, etc.

The problem with building targets as .so files is transparency: you need to know to pass "-load libfoo.so" to llc to get support for the "foo" target. In the long-term, I think this is 100% acceptable. In particular, llc is a low-level developer tool that end-users will ideally not have to directly use in the future. In particular, someday hopefully soon, end-users will interface to the llvm toolchain exclusively through the llvmc tool. llvmc will fork off the subtools when necessary.

I think it's perfectly reasonable for llvmc to know which targets are linked into the llc tool and automatically provide the -load option when needed.

If you accept the above, then we have 3 classes of people to consider: 1. end-users with the above definition, 2. developers that want to test everything [including the nightly testers], and 3. developers who don't want to test everything.

I think that all groups of people are important, the only question is who passes the extra flag. I think that "punishing" group #3 is the right way to go, because they're the ones that want to do something strange, and they're the ones that know enough about llvm to do it. :slight_smile:

-Chris