Hello,
Im trying to build trunk clang in debug version on oldish ubuntu with low mem. Linking lli takes ages and fails on low mem. Is there a chance building would succeed if i used lld instead of ld? If so is there an option either to force lld or whole clang toolchain use in cmake instead of default gcc (both gcc and clang are avail on system)? Otherwise I think ill stick with release.
Best regards,
Pawel Kunio
You can use `LLVM_USE_LINKER=lld` CMake variable to adopt LLD (to build LLVM). And yes, LLD takes less memory and runs faster. Here are some other tips to save memory:
1. You can use `LLVM_PARALLEL_LINK_JOBS=N` (also a cmake variable) to limit the amount of parallel linker jobs to save some memory.
2. Build libraries as shared library via `BUILD_SHARED_LIBS=ON` CMake variable can dramatically speed up the linking time and save you some memory.
3. Since you’re building a Debug build (and you’re building on Linux), `LLVM_USE_SPLIT_DWARF` can dramatically reduce the size of debug info, which, to some extend, also save you some memory during link time.
Best,
-Min
Another way to save disk space and memory would be to build the Release
configuration, and add `-DCMAKE_C_FLAGS=-g1 -DCMAKE_CXX_FLAGS=-g1`
The `-g1` means you get debug info for line numbers, although not really
types and variables; but this is enough for symbolized backtraces after
a crash or assertion. This is what I use normally.
--paulr
Hiya,
Enormous thanks for all suggests and extensive too. Testing splitdwarf asap.
Makes me think what i thought previously. We need dwarf go pdbish way. Separate dblike possibly adressable by url. I was hacking pdb techno before it got sexy. I know a bit how it is organized and could try to help with architecting or designing such solution. Loved dbinfo on vstudio. Gdbish not so much. And i was slaving on my corpo cotton plantation passively using gdb for about a decade.
If catches anyones focus, lets discuss the solution.
Best regards,
Pawel Kunio
czw., 15.04.2021, 23:37 użytkownik Min-Yih Hsu <minyihh@uci.edu> napisał:
Hi,
Aw awesome. Might be spot on for what im looking for. Thanks thanks.
Best regards,
Pawel Kunio
czw., 15.04.2021, 23:41 użytkownik <paul.robinson@sony.com> napisał:
Your best options for that are using DWARF fusion
(-DLLVM_USE_SPLIT_DWARF) or as separate shared libraries
(-DBUILD_SHARED_LIBS). The former will punish the gdb load time and the
latter will penalize execution speed.
Joerg
Hiya,
Enormous thanks for all suggests and extensive too. Testing splitdwarf asap.
Makes me think what i thought previously. We need dwarf go pdbish way.
Split DWARF provides something like this, though with some different
tradeoffs - for instance Split DWARF is easier to use in a distributed
build (since the compiler doesn't have to communicate with a separate
server while it's compiling).
Separate dblike possibly adressable by url.
Symbol servers are another problem - even using classic DWARF you can
then strip the main binary, keep the unstripped binary and put that
unstripped binary on some kind of symbol server system (I'm not sure
if gdb supports something like that natively, but could be done
without DWARF changes).
I was hacking pdb techno before it got sexy. I know a bit how it is organized and could try to help with architecting or designing such solution. Loved dbinfo on vstudio. Gdbish not so much.
What sort of problems have you had with gdb or the DWARF debugging
workflow more generally?
And i was slaving on my corpo cotton plantation passively using gdb for about a decade.
(analogies to slavery aren't suitable for this community)
- Dave
Thank you thank you. Im heading for split dwarf. Im pondering g1 debug options as I dont need gdb very much mostly lineinfos would do for asserts and such.
Best regards,
Pawel Kunio
pt., 16.04.2021, 00:58 użytkownik Joerg Sonnenberger via llvm-dev <llvm-dev@lists.llvm.org> napisał:
As on company remark sorry. Wasnt aware of netiquette. Please rm this msg if need be.
Ok for gdb, load time was major pain. Type casting was patchy some types missing etc, some var tracking especially under optimizations was tricky for example having to cast registers because info this var is stored in this register was missing hmm also one mire thing that could minimize debug info size could be shared types section with removing local copies per compilation unit. Also global vars etc could be stored in shared segment. Also we could think of somehow indexing the db.
My other idea is indeed distributing stripped binaries and devels keeping debug info dbs snapshots locally or via debug servers. If we dont wanna debug servers, stack traces etc should contain stack dumps maybe or some in the middle solution with lineinfo only possibly hashed etc.
Best regards,
Pawel Kunio
pt., 16.04.2021, 01:06 użytkownik David Blaikie <dblaikie@gmail.com> napisał:
Ok for gdb, load time was major pain.
Ah, for large binaries you probably want to compile with
-ggnu-pubnames -Wl,--gdb-index (you must compile with this for Split
DWARF - it relies on the index) that takes, for instance, gdb startup
time debugging clang from 3 minutes to about 3 seconds. (lldb does a
better job lazy loading DWARF - so faster startup time even without an
index (& it can't use the gdb index anyway) trading off some
operations later will be slower as things are lazy loaded later on)
Type casting was patchy some types missing etc,
Hard to say what's at work there - happy to speculate/help if you have
a specific/isolated example.
some var tracking especially under optimizations was tricky for example having to cast registers because info this var is stored in this register was missing
Yeah, that's less about the workflow/technology, but more as we like
to say, the "quality of implementation" - keeping track of variables
under optimization is tricky, and different compilers do it
well/poorly - DWARF certainly offers a good variety of ways to express
these things (and gaining more as DWARF improves - it's certainly not
"complete", but I don't think it ever can be - there'll always be more
cunning optimizations and cunning ways to try to recover the value of
user variables under those optimizations).
hmm also one mire thing that could minimize debug info size could be shared types section with removing local copies per compilation unit.
Yep - that can already be done with DWARF type units
(-fdebug-types-section), but also various strategies to reduce
emitting the types in the first place (hopefully we'll be enabling the
constructor homing strategy by default soon which should reduce
duplicate type information significantly)
Also global vars etc could be stored in shared segment. Also we could think of somehow indexing the db.
Yeah, as mentioned above, gdb (& lld and gold) support the gdb_index
format and that can be generated at link time with -Wl,--gdb-index
(objects need to be compiled with -ggnu-pubnames (that comes by
default if you specify -gsplit-dwarf I think... ) for this to work).
And there is a DWARFv5 debug_names section, but it's not fully
supported yet (we don't have any linker that supports merging
.debug_names from object files into a unified .debug_names in the
linked binary - nor any debuggers that can consume the .debug_names
index).
Is constructor homing this featurette where class di is emitted only for compunits with reference to constructor? Id speculate therell be problems with it as i understand the idea by now. Shared type table feels bit more complex on impl side fir merging etc but feels stronger.
Best regards,
Pawel Kunio
pt., 16.04.2021, 01:40 użytkownik David Blaikie <dblaikie@gmail.com> napisał:
Is constructor homing this featurette where class di is emitted only for compunits with reference to constructor?
Yep - both Clang and GCC already do something equivalent for any type
with virtual functions - the debug info for that type is emitted only
where the vtable is emitted (so called "vtable homing") so it's not
without precedent. Yeah, it means potentially missing any type that
isn't constructed (types used only to hold a bunch of static members,
for instance) - but perhaps such a type isn't needed anyway. This
isn't done for types with trivial constructors - since they don't
generate code, etc.
Id speculate therell be problems with it as i understand the idea by now.
Certainly open to any critiques/bugs/etc. It's currently accessible
under -fuse-ctor-homing
https://www.mail-archive.com/cfe-commits@lists.llvm.org/msg218269.html
Shared type table feels bit more complex on impl side fir merging etc but feels stronger.
Tradeoffs to be sure - anything that's shared during compile time's
not really usable in the build environment (massively distributed) I
support, but can work for others (like MSVC does). C++ modules (both
Clang Header Modules and C++20 standard modules) offer another way to
"home" debug info - since modules are then a first class build product
they can be used to home debug info quite reliably (template
instantiations may be duplicated - since they may be instantiated in
multiple unrelated places, but even then any module depending on a
module with an insntantiation need not re-do that instantiation).
- Dave
Hi,
Also if that matters on winxp mingw x8632 arch debug command responsiveness was measured in tens/hundreds of milis while same in vc it was in few nanos approximately and it was common for most archs. Seemed like gdb antifeature more than trap exception handling kind of problem but speculating here.
Best regards,
Pawel Kunio
pt., 16.04.2021, 01:06 użytkownik David Blaikie <dblaikie@gmail.com> napisał:
Hello,
Not sure i have anything useful to say on discussion on ctorhoming but should it be so ill try to chime in on discussion. On bugs i cant talk. Didnt heavily test it or use it much at all but i could say how it may seem when trying to compete with vc and how it may look from john does sw devel side possibly not a superstar one.
Best regards,
Pawel Kunio
pt., 16.04.2021, 02:03 użytkownik David Blaikie <dblaikie@gmail.com> napisał:
Thanks everyone for great advice, maybe we should mention some of this on the https://llvm.org/docs/GettingStarted.html page.
Maybe under a subtitle like “Development on low-end devices” or something. This would be helpful for newcomers who are working with low-end devices.
Also since it was not mentioned, ccache can help with build times during development.
pawel k. via llvm-dev <llvm-dev@lists.llvm.org>, 16 Nis 2021 Cum, 03:10 tarihinde şunu yazdı:
Hello,
Splitdwarf worked nicely for me. Big thaks to all contributing and biy there were many useful hints.
I think that would be useful. I can try to collect them all and brief each and then somebody could apply the styling if need be or if just pure text paragraph would suffice I can prepare some file like that. I especially liked lineinfo+funcaddrs only option. Very useful for stacktraces and asserts if not planning to use gdb.
Hope to help somehow.
Big thanks and best regards,
Pk
pt., 16.04.2021, 14:18 użytkownik kuter dinel <kuterdinel@gmail.com> napisał:
Hello,
Splitdwarf worked nicely for me. Big thaks to all contributing and biy there were many useful hints.I think that would be useful. I can try to collect them all and brief each and then somebody could apply the styling if need be or if just pure text paragraph would suffice I can prepare some file like that. I especially liked lineinfo+funcaddrs only option. Very useful for stacktraces and asserts if not planning to use gdb.
I think most of this is written up here:
https://llvm.org/docs/GettingStarted.html#common-problems
Though looks like it could have a bullet point for -g1-for-backtraces too.