HELP! Recent failure on llvm buildbot

Project Infrastructure LLVM Dev List Archives
1

I'm working on lldb.

I've just submitted a very small change (r217229) to Triple.h/.cpp. Soon after I get a mail

Details:
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/2571
Blamelist: mg11

My small change certainly did not cause lldb's build to fail on my machine. I looked into the build-log:

http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/2571/steps/build_Lld/logs/stdio

It wasn't obvious how my change resulted in failure.

When I tried to replicate the build of the last 2 targets listed in the log, I did see this, on my machine:

$ ninja LLVMCodeGen
[15/15] Linking CXX shared library lib/libLLVMCodeGen.so

$ ninja LLVMMipsCodeGen
ninja: error: unknown target 'LLVMMipsCodeGen'

Can anyone help?

thanks
Matt

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.

2

Hi Matthew,

My small change certainly did not cause lldb's build to fail on my machine.
I looked into the build-log:

It's probably just lld building with -Werror (and likely on a
different compiler than you too, or you'd have seen the warning). I've
fixed this one up in r217233, we'll see what that does to the lld
builder.

When I tried to replicate the build of the last 2 targets listed in the log,
I did see this, on my machine:

That's usually not particularly useful even if there are targets for
them. The builds are very parallel and it's difficult to work out just
which target actually caused the problem "ninja check-all" should be
fine for most cases.

Cheers.

Tim.

3

Hi,

The build log show this error:

/Users/buildslave/as-bldslv9/lld-x86_64-darwin13/llvm.src/lib/Target/ARM/MCTargetDesc/ARMMCTargetDesc.cpp:92:11: error: enumeration values 'KalimbaSubArch_v3', 'KalimbaSubArch_v4', and 'KalimbaSubArch_v5' not handled in switch [-Werror,-Wswitch]
  switch (triple.getSubArch()) {

This is a warning turned into an error by -Werror. LLVM’s code should be warning free.
You indeed added these entries to the enum and they seem not to be handled in ARMMCTargetDesc.cpp. If you don’t know how to add these to ParseARMTriple(), I’d suggest you revert your change until someone with ARM MC knowledge helps you out.

Fred

4

Hi Tim,

Thanks for helping me with this. The route which I take to build lldb (cmake/ninja on Fedora) seems (irritating) permissive of warnings. So I missed this when I originally submitted my change.

I'll certainly bear this in mind if I make any more llvm changes!

thanks
Matt

Tim Northover wrote:

Hi Matthew,

My small change certainly did not cause lldb's build to fail on my machine.
I looked into the build-log:

It's probably just lld building with -Werror (and likely on a
different compiler than you too, or you'd have seen the warning). I've
fixed this one up in r217233, we'll see what that does to the lld
builder.

When I tried to replicate the build of the last 2 targets listed in the log,
I did see this, on my machine:

That's usually not particularly useful even if there are targets for
them. The builds are very parallel and it's difficult to work out just
which target actually caused the problem "ninja check-all" should be
fine for most cases.

Cheers.

Tim.

  To report this email as spam click https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ== .

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.

5

Hi Fred,

Thanks. How did you get to

"
The build log show this error:

/Users/buildslave/as-bldslv9/lld-x86_64-darwin13/llvm.src/lib/Target/ARM/MCTargetDesc/ARMMCTargetDesc.cpp:92:11: error: enumeration values 'KalimbaSubArch_v3', 'KalimbaSubArch_v4', and 'KalimbaSubArch_v5' not handled in switch [-Werror,-Wswitch]
   switch (triple.getSubArch()) {

"

from here
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/2571

Sorry to seem lame, but I could have fixed it myself if I'd known how to navigate to that line describing the failed invocation.

thanks
Matt

Frédéric Riss wrote:

6

Hi Fred,

Thanks. How did you get to

"
The build log show this error:

/Users/buildslave/as-bldslv9/lld-x86_64-darwin13/llvm.src/lib/Target/ARM/MCTargetDesc/ARMMCTargetDesc.cpp:92:11: error: enumeration values ‘KalimbaSubArch_v3’, ‘KalimbaSubArch_v4’, and ‘KalimbaSubArch_v5’ not handled in switch [-Werror,-Wswitch]
switch (triple.getSubArch()) {

"

from here
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/2571

Sorry to seem lame, but I could have fixed it myself if I’d known how to navigate to that line describing the failed invocation.

There are no dumb questions. My answer might seem a bit disappointing though :slight_smile: I just scrolled quickly though the log. The errors appear in red, they were easy to spot. Depending on the build system used, you can also search for specific patterns (like *** for make) in the log.

Fred

7

There are no dumb questions. My answer might seem a bit disappointing though
:slight_smile: I just scrolled quickly though the log. The errors appear in red, they
were easy to spot. Depending on the build system used, you can also search
for specific patterns (like *** for make) in the log.

Specifically, it's the "stdio" link under the red failing step in this
case. It's often not obvious where the useful information is though, I
still often end up clicking randomly until I find it.

A good thing to search for in build failures is "error:". Lots of
places can mention "error" but the extra colon is pretty
characteristic of compiler diagnostics.

Cheers.

Tim.

8

I'm now being blamed by a buildbot for this

  http://lab.llvm.org:8011/builders/clang-native-arm-cortex-a9/builds/21468

but inspecting the audit trail, reveals that this is test failure which emanates from a (slow?) build which took over 4 hours to complete.

I believe that this issue is historic and is fixed by r217233.

Matt

Tim Northover wrote:

There are no dumb questions. My answer might seem a bit disappointing though
:slight_smile: I just scrolled quickly though the log. The errors appear in red, they
were easy to spot. Depending on the build system used, you can also search
for specific patterns (like *** for make) in the log.

Specifically, it's the "stdio" link under the red failing step in this
case. It's often not obvious where the useful information is though, I
still often end up clicking randomly until I find it.

A good thing to search for in build failures is "error:". Lots of
places can mention "error" but the extra colon is pretty
characteristic of compiler diagnostics.

Cheers.

Tim.

  To report this email as spam click https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ== .

Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom
More information can be found at www.csr.com. Keep up to date with CSR on our technical blog, www.csr.com/blog, CSR people blog, www.csr.com/people, YouTube, www.youtube.com/user/CSRplc, Facebook, www.facebook.com/pages/CSR/191038434253534, or follow us on Twitter at www.twitter.com/CSR_plc.
New for 2014, you can now access the wide range of products powered by aptX at www.aptx.com.

9

Hi Matt,

Don't worry, no one is "blaming" you. :slight_smile:

Builds fail for various reasons, sometimes it's your commit, sometimes
it's a commit on a repository that is not tracked by that bot,
sometimes it's a bot issue.

In this case, the bot is unstable and I'm working to get us more
stable ARM bots as we speak, but for the time being, the rule of thumb
is this:

When you get a mail from a buildbot failure, you:

1. Click on the link and inspect the error. If it's obviously you,
please fix it asap or revert the patch. Also, ping the IRC channel and
reply to the email (which will have arrived to all other committers)
that you're looking into it, so they don't have to worry.

2. If it's not obviously you, check:
2a. The stdio output of the red bits for timeouts, internal compiler
error, no space left on disk, lost connection. These are bot problems,
ignore for now, but keep an eye to make sure the next builds are
green.
2b. Or, the tests failing, and if the test was added or changed by
another commit on the same build, feel free to reply the email asking
the developer to look into it.
2c. Or, that the failure can be reproduced in your machine and try to
see if there's anything you did that could come to that.

If it was your commit, sometimes, you'll need help from the
maintainer, for instance, if this is an ARM specific failure.

Every bot has an owner and, if it's your fault, you should email the
owner if he hasn't contacted you yet. Not everyone does that, and bot
owners know that they should wait a bit to spot if any "fixing bots"
commit is coming through before calling foul. Though, we generally
mail you anyway, just to make sure you are aware of the problem.

Some of us try to bisect on our own hardware and will pinpoint the
offending commit and work with the developer to fix it. Some of us may
even provide you access to a board so you can debug your own code on
our architecture.

If that takes more than a few days, reverting the commit is probably
best. You should revert your own commits, or work with whomever is
reverting it, to make sure nothing is removed by accident (multiple or
old commits are harder to revert).

If that takes even longer, you should create a bugzilla report,
copying the owner of the bot, architecture code owner, etc and work
from there.

If that feature/bugfix is *really* important to you, please make
everyone aware by marking the bug as so, and display interest and
pro-activity when dealing with the fixes and investigation. Also, let
us know if that needs back-porting to the current releases' patch
release (3.5.1 etc) and contact the point-release manager, which may
change, so enquire on the list.

Hope that helps,

cheers,
--renato

10

Hi Renato

Thanks for laying out the strategy.... but yeah, when I said "blame" I was just repeating the buildbot's message. I don't take offense too easily... :wink:

Matt

Renato Golin wrote: