Disk space and RAM requirements in docs

Hi,

Have anybody recently built LLVM in Debug mode within space requirements from the Getting Started doc?

An LLVM-only build will need about 1-3 GB of space. A full build of LLVM and Clang will need around 15-20 GB of disk space.

Apologies for the long snippet, but here is top 20 lines from du -h | sort -rh on my FreeBSD directory (full log is attached):

91G build
40G build/bin
25G build/tools
25G build/lib
24G build/tools/clang
15G build/tools/clang/unittests
5.0G build/tools/clang/lib
3.9G build/lib/Target
3.1G build/tools/clang/unittests/Tooling
3.1G build/tools/clang/test
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx/bin
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp
2.8G build/tools/clang/test/Tooling/Output
2.8G build/tools/clang/test/Tooling
1.9G build/tools/clang/unittests/Frontend
1.6G build/tools/clang/unittests/Driver
1.5G build/tools/clang/unittests/ASTMatchers
1.4G build/tools/clang/lib/StaticAnalyzer
1.3G build/lib/CodeGen

Note that Clang’s unittest and test directory together occupy more than 18 GB (which is about the size manual assigns for the entire build), with most of the space in test taken up by output of tooling tests (is that for MacOS, by the way?). Also, looks like five of the unittest directories for clang components are taking more than a gigabyte each (with Tooling using more than 3 GB).

Is this expected or am I doing something wrong? If it is expected, should the manual be fixed?

There are no RAM requirements in the doc, from my observation linker process on Linux or FreeBSD needs about 10 GB at the moment (and this has been growing for a while). Should this be also reflected in hardware requirements as well?

And lastly is there a way to reduce the space consumption by Clang unittests and Tooling tests aside of disabling those?

Best,

Petr

build.du (198 KB)

It's been like this for a while, and IMO it's a bug in how those tests
were implemented. Disk space is cheap, but shouldn't be wasted that
badly.

Unfortunately the fix isn't simple. I tried modifying the tests to
remove their output when done a while back, but Windows has weird
non-determinism in when locks on just-closed files are released and
wouldn't let the script remove them sometimes; so it had to be
reverted.

I think someone suggested modifying llvm-lit to keep closer track of
temporaries and remove them, but I don't recall how that necessarily
helps things and never got around to implementing it.

Cheers.

Tim.

Hi,

Have anybody recently built LLVM in Debug mode within space requirements from the Getting Started doc?

https://llvm.org/docs/GettingStarted.html#hardware

> An LLVM-only build will need about 1-3 GB of space. A full build of LLVM and Clang will need around 15-20 GB of disk space.

From my experience this numbers looks drastically low. On FreeBSD my recent builds consumed more than 70 GB and on Linux - more than 60. I am building LLVM, Clang, and LLD with all default targets and default settings. On FreeBSD I am also building Polly. Also, building `clang-check` target adds around to 20 GB to the build directory size.

Apologies for the long snippet, but here is top 20 lines from `du -h | sort -rh` on my FreeBSD directory (full log is attached):

91G build
40G build/bin
25G build/tools
25G build/lib
24G build/tools/clang
15G build/tools/clang/unittests
5.0G build/tools/clang/lib
3.9G build/lib/Target
3.1G build/tools/clang/unittests/Tooling
3.1G build/tools/clang/test
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx/bin
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp/mock-libcxx
2.8G build/tools/clang/test/Tooling/Output/clang-check-mac-libcxx-fixed-compilation-db.cpp.tmp
2.8G build/tools/clang/test/Tooling/Output
2.8G build/tools/clang/test/Tooling
1.9G build/tools/clang/unittests/Frontend
1.6G build/tools/clang/unittests/Driver
1.5G build/tools/clang/unittests/ASTMatchers
1.4G build/tools/clang/lib/StaticAnalyzer
1.3G build/lib/CodeGen

Note that Clang's `unittest` and `test` directory together occupy more than 18 GB (which is about the size manual assigns for the entire build), with most of the space in `test` taken up by output of tooling tests (is that for MacOS, by the way?). Also, looks like five of the unittest directories for clang components are taking more than a gigabyte each (with Tooling using more than 3 GB).

Is this expected or am I doing something wrong? If it is expected, should the manual be fixed?

There are no RAM requirements in the doc, from my observation linker process on Linux or FreeBSD needs about 10 GB at the moment (and this has been growing for a while). Should this be also reflected in hardware requirements as well?

I think it would be worth strongly recommending the use of LLD or gold
to link. I've helped out a number of students/newcomers getting LLVM
to build for the first time and encountering issues due to binutils ld
OOMing is a common issue.

And lastly is there a way to reduce the space consumption by Clang unittests and Tooling tests aside of disabling those?

Setting -DBUILD_SHARED_LIBS=True will reduce space requirements by
using shared libraries rather than static libraries. It will impact
the time taken to run LLVM and Clang tests though, due to greater
startup time.

Best,

Alex

And lastly is there a way to reduce the space consumption by Clang unittests and Tooling tests aside of disabling those?

Setting -DBUILD_SHARED_LIBS=True will reduce space requirements by
using shared libraries rather than static libraries. It will impact
the time taken to run LLVM and Clang tests though, due to greater
startup time.

Presumably it'll also increase runtime due to ELF interposing requirements.

LLVM_BUILD_LLVM_DYLIB, which builds the large monolithic shared library, could be better on that front if symbol visibility is set correctly (which I haven't double-checked).

Cheers,
Nicolai

It looks as if running the tests is adding a lot of space to your builds. I have three LLVM trunk build directories on this machine (FreeBSD):

Release, building clang, clang-tools-extras, and lld: 1.2GB
Debug, building clang, clang-tools-extras, and lld: 36GB
Debug + ASan, building clang, lldb, and lld: 40GB

1-3GB is still enough to do a release build and not run the tests. I don't think I've ever done a 'ninja check' in any of these (I have other build directories for things I'm actively working on), so you can get down to under 40GB without running the tests for a Debug build.

I'd definitely recommend following Chandler's advice though and not bothering with a debug-without-asan build. Debug builds are sufficiently slow that the overhead of Asan on top is not very big - working on a release build for fast testing and then switching to Asan + Debug for debugging is a nice way of working.

David

Thank you for all the replies.

I deleted the build directory and re-ran CMake from scratch, still got to 70 GB on FreeBSD. My Linux debug build is around 40 GB, which is what is everyone seem to expect, so I have two questions:

- Would it be reasonable to update the Getting Started document to bump up the size of statically linked debug build to 30-40 GB?

- I am also very curious about the size disparity between the build on my machine and David's, any pointers would be greatly appreciated.

FreeBSD defaults its system compiler to use -fstandalone-debug - this
option produces significantly more debug info per object file (though
if you use DWARF type units, -fdebug-types-section, at least the
executables won't be any larger with/without -fstandalone-debug) so
that the resulting binaries can be successfully debugged with LLDB.

If you don't plan to use LLDB, I'd suggest disabling this feature by
adding -fno-standalone-debug somewhere in your compiler arguments.

I'd also suggest using Split DWARF (there's a CMake variable for that
- I forget the exact name, but searching your CMakeCache.txt for
"SPLIT" should find it quick enough) as that'll reduce the debug info
size impact significantly (with the tradeoff that now the executable
will have a dependency on other files with debug info (more like the
MachO/Apple model of debug info distribution) - so you can't take the
executable & delete your build tree and still debug the binary).

It would be great if our build system could default to gold or LLD if available.