Failures on clang-with-lto-ubuntu builders

Hi,

Both the clang-with-lto-ubuntu and clang-with-lto-ubuntu-release builders are
consistently failing due to one of the linker jobs running out of memory.
Is there any easy way to fix these failures (Maybe by using zram?)
If not, can we disable these bots?

Thanks,
Tom

best to check the bot owners & include them on an email like this? But yeah, if they aren’t responsive to stabilizing their bots - certainly open to disabling them straight on the zorg repo.

best to check the bot owners & include them on an email like this? But yeah, if they aren't responsive to stabilizing their bots - certainly open to disabling them straight on the zorg repo.

Is there a list of bot owners somewhere? I tried to dig through the llvm-zorg
repo to figure out who owned the bot, but I couldn't find anything.

-Tom

best to check the bot owners & include them on an email like this? But yeah, if they aren't responsive to stabilizing their bots - certainly open to disabling them straight on the zorg repo.

Is there a list of bot owners somewhere? I tried to dig through the llvm-zorg
repo to figure out who owned the bot, but I couldn't find anything.

I've learned that there are instructions for finding bot owners in the
Developer Policy: https://llvm.org/docs/DeveloperPolicy.html#working-with-the-ci-system

It looks like Galina is the owner of these workers.

-Tom

Hi Tom,
That looks like great quick fix, Im wondering if we could propose data format or computation model change for example to db- or other disk-vs-mem-based where we could lto link huge projects anyway. I wonder whether whole lto data needs to be loaded into mem at once and whether its tree- or other datastructure based.

-Pawel

wt., 11.05.2021, 07:26 użytkownik Tom Stellard via llvm-dev <llvm-dev@lists.llvm.org> napisał:

Hi Tom,
That looks like great quick fix, Im wondering if we could propose data format or computation model change for example to db- or other disk-vs-mem-based where we could lto link huge projects anyway. I wonder whether whole lto data needs to be loaded into mem at once and whether its tree- or other datastructure based.

That’s what ThinLTO is intended to/does address - but has tradeoffs. It has limited whole program data - but better than traditional compilation, since it can use profiling/other analysis to inform how to partition the program.

  • Dave

Cool. Thanks for letting me become less ignorant. Im still discovering llvm and I pretty much love what I saw so far. Llvm+clang is a small wonder of the world.

-Pawel

wt., 11.05.2021, 09:15 użytkownik David Blaikie <dblaikie@gmail.com> napisał:

Thanks, Tom!

The both builders are back to normal and should work reliably.

The memory usage during the build grew up for about 15% because of https://reviews.llvm.org/rGb5c63e30ca1af7e36ba5069eb0121d1eb4b06ebb, and pushed the builders over the edge.

Galina

Thanks, Tom!

The both builders are back to normal and should work reliably.

The memory usage during the build grew up for about 15% because of https://reviews.llvm.org/rGb5c63e30ca1af7e36ba5069eb0121d1eb4b06ebb, and pushed the builders over the edge.

Yevgeny - this seems like a significant increase in memory usage. Is that intended? What’s the general motivation/justification for the change?