Proposal: Enable BUILD_SHARED_LIBS=ON by default for debug build.

Project Infrastructure LLVM Dev List Archives
1

Hello,

I understand shared linked is currently a lot slower than static linked.

However, for debug build, where speed is not required. Shared link has

A lot of advantage.

Static linked clang/llvm on linux is 17G, shared build is only 1.5G. On a mainstream

linux host machine with 12G memory, only –j1 can be used to finish final linking step.

With –j8 the linux will link the linking. With 17G footprint, copying from build folder

to install folder is also very slow. So it is very reasonable to set

BUILD_SHARED_LIBS=ON as default for debug build because it would improve usability

and performance.

Thanks,

Yin

2

Hello,

I understand shared linked is currently a lot slower than static linked.

However, for debug build, where speed is not required. Shared link has

A lot of advantage.

Static linked clang/llvm on linux is 17G, shared build is only 1.5G.

Assuming you're talking about a debug build, you can probably get that down
a lot with at least -fdebug-types-section or with -gsplit-dwarf.

On a mainstream

linux host machine with 12G memory, only –j1 can be used to finish final
linking step.

If you use Ninja, I believe it has a separate pool limit you can tweak for
linking (but, again, with -fdebug-types-section or -gsplit-dwarf, you might
make the link steps more palatable (-gsplit-dwarf is probably the
better/more important choice, though))

But I'd be surprised if the limit is that low - are you using gold? Or
binutils ld? Gold is a good/better choice here, and usually I only end up
using about a GB of RAM per linking process I Think.

With –j8 the linux will link the linking. With 17G footprint, copying from
build folder

to install folder is also very slow.

Solid state, on spinning disks? Solid state would help a bit.

So it is very reasonable to set

BUILD_SHARED_LIBS=ON as default for debug build because it would improve
usability

and performance.

The thing is, the performance tradeoff also hits test execution time. At
least last time I checked (granted, on a 32 core, 64GB of RAM machine) the
build time improvement (on a clean build) with shared libs was not saved by
a single execution of the regression test suite (which took longer than the
savings). Let alone on an incremental build... if I remember my numbers
correctly.

Not sure whether it's the right call to change the default - people can
change their own settings easily enough.

- Dave

3

Hi David,

I assume you have a powerful machine. Our drive space is on network mounted

machined by IT department. The machine is default Ubuntu setup with 8 cores.

Shared build is showing huge advantage due to size reduction. I will try those two

debug options to see how it works.

Thanks,

Yin

4

Hi David,

I assume you have a powerful machine. Our drive space is on network mounted

machined by IT department. The machine is default Ubuntu setup with 8
cores.

It helps a lot to build on a non-NFS scratch disk. Is your /tmp network mounted too? If not, I'd suggest building there.

Jon

5

Yeah, building on a network attached storage would be pretty painful, I
imagine.

You can of course use a shared build yourself, it may be better for your
use/scenarios of course - I'm just pointing out that there's lots of us for
which it isn't the best option, enough so that I wouldn't be confident
about changing the default.

6

Hi David,

It is O.K. I just hope BUILD_SHARED_LIBS=ON is tested when a patch is

merged. Yesterday, I found the tip failed to build with ON. Today, tip is

O.K to build in shared linking.

Yin

7

Hi David,

It is O.K. I just hope BUILD_SHARED_LIBS=ON is tested when a patch is

merged. Yesterday, I found the tip failed to build with ON. Today, tip is

O.K to build in shared linking.

I'm not sure we have (m)any buildbots running with a shared link on. Though
we do tend to catch any dependency problems like the one you saw soon-ish
(within a day or two) & patches are always welcome for that sort of thing
(as for other cleanup on configurations that are less frequently used).

- Dave

8

Please don’t turn this on by default (even for Debug builds). On Darwin link times for debug builds really aren’t bad, but BUILD_SHARED_LIBS makes running check WAY WAY WAY slower.

-Chris

9

Hi Yin,

I second David's words. I use shared libs myself on debug builds, but
static builds with gold on a local disk is pretty feasible. Before I
used gold, I had to have 16GB of RAM on my laptop, now I need less
than 8GB for static builds.

It is O.K. I just hope BUILD_SHARED_LIBS=ON is tested when a patch is
merged. Yesterday, I found the tip failed to build with ON. Today, tip is
O.K to build in shared linking.

This is a different matter, and as David said, one that may not be
*that* critical, given to the number of people that use shared libs on
debug builds.

Chances are that ToT is ok now because someone fixed whatever the bug
was, and not because of sheer luck. Don't worry too much about it.

I see no reason to turn shared libs on by default on debug builds,
given that it's just one command line option in CMake. Whatever the
default state of that flag is, some people will have to have it,
others won't. If you change the default, you'll force *everyone* to
change their flags for no reason.

cheers,
--renato

PS: Life is short, don't build stuff over NFS. :slight_smile:

10

Hi,

I enabled BUILD_SHARED_LIBS=ON for the llvm-mips-linux builder when I switched it over to cmake.

11

Ah, handy - thanks!

12

Hi Golin,

Thank for your information. I will upgrade our host toolchain to improve building performance.

Thanks,

Yin

13

Yin,
     You might want to revisit this issue by trying
-DLLVM_LINK_LLVM_DYLIB=ON (which isn't darwin specific cmake option
despite its name). With the commit of r260641 in both trunk and 3.8
branch, all of the linkage glitches causing test suite regressions are
now eliminated so the unified libLLVM shared library is now usable.
             Jack