Optimized subarch library support

I'd like to add support to Clang for picking subarch-optimized versions of libraries. One place I see this being particularly useful is on x86 where there are a handful of different vector units available but libraries need to be built to the lowest common denominator and therefore can't take advantage of the increased performance of architectures supporting them (runtime cpu dispatch aside). Embedded arm targets could benefit from this as well.

GCC's multilibs are one possible solution to this problem, and they seem relatively orthogonal to the multiarch movement, so their structure is probably a reasonable place to start.

For example, on x86 it could look something like this:
/lib/i386-linux-gnu/
/lib/i386-linux-gnu/sse2/
/lib/i386-linux-gnu/sse4.2/
/lib/i386-linux-gnu/avx/
and then at compile time, depending on which flags (target, march, mcpu, etc) were provided, Clang would pick the appropriate directory suffix and link against what's inside it. This would give the user the best thing that is compatible with their flags, rather than always sticking with the (s)lowest commmon denominator.

There are two parts to this as far as implementation goes: 1) library discovery, and 2) picking the best one for a given set of flags.

In order for (1) to work, these directory names would either have to be standardized, or have some sort of configuration file that says which features each subdir supports. Given that there are so many combinations of flags that could affect this library choice, I think it makes most sense to go with config files. OTOH, this would require support from distros to put in the right config file in order to get it to 'just work' when Clang is installed, so that feels slightly less desirable. I'm not sure the right way to go here.

Part (2) can be further split up into determining compatibility and assigning rank, both of which seem like architecture family specific things that could be determined from the feature sets mentioned above.

Implementation-wise, I'm thinking of something roughly like this:

class Multilib {
public:
   Multilib(StringRef Suffix);
   Multilib(StringRef Suffix, ArrayRef<std::string> FeatureSet);
   bool SupportsFeature(StringRef Feature);
   std::string GetSuffix() { return Suffix; }
private:
   std::vector<std::string> FeatureSet;
   std::string Suffix;
};

class MultilibManager {
public:
   std::vector<Multilib> FindCompatibleMultilibs(StringRef Path, llvm::Triple Triple, const ArgList &Args);
   Multilib GetBestMatch(ArrayRef<Multilib> Libs, llvm::Triple Triple, const ArgList &Args);

private:
   void AddConfigFileLibs(StringRef ConfigFilePath, ArrayRef<Multilib> Libs);
   bool IsCompatible(const Multilib & Lib, llvm::Triple Triple, const ArgList &Args);
   bool IsSuperior(const Multilib & a, const Multilib & b, llvm::Triple Triple, const ArgList &Args);
};

appropriately hooked into Driver/Toolchains.cpp

Comments & suggestions before I put together a more formal patch would be much appreciated :slight_smile:

Selecting libraries for a specific processor seems more of a load-time decision than a compile-time one. If you're building everything tuned for a specific processor, why would you install libraries that are not so tuned? And isn't the current direction to use run-time linker tricks to automatically select the most optimal implementation from a variety of options within a single library, rather than use different libraries?

David

FYI

The Clang driver supports MIPS target multilib selection for a couple
of toolchains including Codesourcery. All paths (relative to the
toolchain root) are hard coded into the driver and it selects the best
match using provided command line arguments.

This test case demonstrates running the Clang driver with Codesourcery
MIPS toolchain:
tools/clang/test/Driver/mips-cs.cpp

As to the configuration file - you can take a look at this discussion:
http://clang-developers.42468.n3.nabble.com/Clang-universal-driver-project-state-td4033598.html

In general I like the idea to implement more general solution covers
not only MIPS target.

Simon

Selecting libraries for a specific processor seems more of a load-time decision
than a compile-time one.

Not always.

If you're building everything tuned for a specific processor, why would you
install libraries that are not so tuned?

You may not be installing much of anything. Everything could be statically
linked. This is quite common in bare-metal applications for embedded systems.
The application is built into one monolithic blob and the most optimal standard
library for the given toolchain options is chosen at application build time.

With GCC this is accomplished using multilibs. We are looking at how to
provide similar functionality with Clang. Note that multilib support is
actually listed as one of the goals for the Clang universal driver:
http://clang.llvm.org/UniversalDriver.html. Not sure how up-to-date that is,
though :slight_smile:

And isn't the current direction to use run-time linker tricks to automatically
select the most optimal implementation from a variety of options within a single
library, rather than use different libraries?

In general, I don't know. I'll have to research it, but I don't this think is
always a practical solution for bare-metal type environments and in embedded
GNU/Linux GCC type environments folks still do use multilibs with separately
tuned sysroots.

Sorry, saying it is a "goal" is overstating things. It is only mentioned as a
related work on that page.