No new builds going through buildbot CI for 11 hours

@gkistanova, I am looking at https://lab.llvm.org/buildbot/#/console and the last commit is from 11 hours ago. Can you take a look?

Any update on the situation here please? There have been no builds for approaching a day now

I’m looking at this. Will let you know when the problem is resolved (hope soon).

1 Like

Buildbot is back to normal.
Please ping me if you will see issues.

It will be catching up with the collected build requests for a while. I’ll clean build queue for slower builders to help with this.

Later today I will take the lab down for some maintenance, will post an announce when will know the exact time.

1 Like

@gkistanova, there is some weirdness in that https://lab.llvm.org/buildbot/#/builders would show some builds in-progress/done for some bots, but the page for the builder (e.g., https://lab.llvm.org/buildbot/#/builders/57) do not have the newer builds.

@reinterpretcast, this seems a buildbot UI issue. It requests a certain number of records, and the logic does not work well when build queues are long, like what it was when you were looking there.

It should be good now, since the majority of the build requests backlog has been processed.

Thanks @gkistanova for the analysis. I’m still having that UI issue on my end, but now that I know about it, I won’t be confused by it.

Hm. That’s strange.

Here is what I see on the builders screen:

And here is the worker:

The builder screen also looks consistent to the other two:

If you see something else with respect to different time of requests, maybe try to clean a browser cache?

I’m not seeing the issue right now. I’ll try that next time – thanks.