Need help reading the LLVM Buildbot results

From r167602 and onwards I get a fail in ‘make check-all’ for llvm+clang, built with gcc-4.7.1 (but not with clang-3.1) on Linux x86_64:

Failing Tests (1):
Clang :: CodeGenCXX/mangle-ms-templates.cpp

clang: /dev/shm/uabpath/master/include/llvm/Support/Casting.h:208: typename llvm::cast_retty<To, From>::ret_type llvm::cast(const Y&) [with X = clang::TemplateSpecializationType; Y = const clang::Type*; typename llvm::cast_retty<To, From>::ret_type = const clang::TemplateSpecializationType*]: Assertion `isa(Val) && “cast() argument of incompatible type!”’ failed.

This made me take a look at the LLVM Buildbot pages at http://lab.llvm.org:8011/. However, I find it difficult to find useful information matching this problem.

  • Can I look up results for r167602 (or closely afterwards)? (Can I compare it with r167601 or less?)
  • How can I see if some buildbot is using gcc-4.7.1 (or later)?
  • There seems to be quite a few failed builds in the buildbots. Which buildbot configurations should I expect to be maintained to produce mostly non-failed builds?
  • Is there some general introduction document for reading the LLVM Buildbot results?

Regards,
Patrik Hägglund

From r167602 and onwards I get a fail in 'make check-all' for llvm+clang,
built with gcc-4.7.1 (but not with clang-3.1) on Linux x86_64:

Failing Tests (1):
    Clang :: CodeGenCXX/mangle-ms-templates.cpp

clang: /dev/shm/uabpath/master/include/llvm/Support/Casting.h:208: typename
llvm::cast_retty<To, From>::ret_type llvm::cast(const Y&) [with X =
clang::TemplateSpecializationType; Y = const clang::Type*; typename
llvm::cast_retty<To, From>::ret_type = const
clang::TemplateSpecializationType*]: Assertion `isa<X>(Val) && "cast<Ty>()
argument of incompatible type!"' failed.

If you take the full command line out of the build log & run it
separately you should be able to reproduce this assert. Depending on
the complexity of the test, you may be able to simply remove the
"-cc1" that's probably in the command line & watch it fail but this
time you'll get a crash report from Clang including the path to a .sh
file containing the command line and a .cpp file containing the
preprocessed source code necessary to reproduce the problem. If you
can file that in a bug we should be able to reproduce & fix it.

This made me take a look at the LLVM Buildbot pages at
http://lab.llvm.org:8011/. However, I find it difficult to find useful
information matching this problem.

Don't worry, you're not the only one. This area could use some
improvement (& there's some amount of work going on in that area)

* Can I look up results for r167602 (or closely afterwards)? (Can I compare
it with r167601 or less?)

I don't believe so - I think only a certain sliding window of logs are
kept (& I don't know of an easy way to view a particular revision
across the bots either)

* How can I see if some buildbot is using gcc-4.7.1 (or later)?

I don't believe so - some bots indicate something of their
purpose/role in the name of the builder, but it's not a well specified
attribute. I suppose we could add something to the builder profile
that causes us to run a version check on the host compiler (but that
would only show up in the build logs at best, not in the builder
description). Anything else I think would be manually maintained
(which might still be worthwhile)

* There seems to be quite a few failed builds in the buildbots. Which
buildbot configurations should I expect to be maintained to produce mostly
non-failed builds?

Unclear - generally you can look at the finite history shown on the
buildbot console page & if a bot's been red for that entire time it's
probably about worth ignoring (until someone takes the time to clean
it up). We should move the dead buildbots aside until someone revives
them, rather than leaving them there adding noise to the page.

* Is there some general introduction document for reading the LLVM Buildbot
results?

Not that I know of.

Thanks David,

file that in a bug

I have filed TR 14321. http://llvm.org/bugs/show_bug.cgi?id=14321

(Just removing -cc1 altered the behavior.)

Don't worry, you're not the only one.

Good. :slight_smile:

/Patrik Hägglund

That'd be a very nice feature!

--renato

I don't believe so - I think only a certain sliding window of logs are
kept (& I don't know of an easy way to view a particular revision
across the bots either)

For me a bigger issue has been inability to see a long list of results for a
particular builder (say, arm-cortex-a9-blah). Unfortunately while I can see
"the last 20 builds over all targets" I can only see "the last five builds
for a particular builder". What's frustrating is that I'm pretty sure the
buildbot is keeping the info, I just can't see how to get it.

The URL's for build info are very regular in how they are constructed. You
can just guess what the URL for "100 builds ago is", and it works!

Ciao, Duncan.

When you're on a builder's page (eg:
http://lab.llvm.org:8011/builders/clang-x86_64-debian ) you can "view
in waterfall" (
http://lab.llvm.org:8011/waterfall?show=clang-x86_64-debian ) which
gives you more history (even on the first page it's got about 14
builds & it has multiple pages to go back further) but it's the high
detail waterfall view, not a simple red/green build status summary of
each build.

Having more history in the "recent builds" would be nice (or somewhere
else where that could be easily visualized)

Also, for what it's worth, the buildbot config can be changed to some
constant other than '5' for the recent builds list, but it would still
be only a constant across all builders. From the buildbot docs:

/builders/$BUILDERNAME
  This describes the given Builder and provides buttons to force a
build. A numbuilds=argument will control how many build lines are
displayed (5 by default).

Thanks. I'd made the wrong assumption "show in waterfall" was a link back to
the multi-builder waterfall page. That's very helpful to know.