I think this would be the ideal solution and already a big improvement over what happens currently. Would just writing this out at the end of the compile job be a reasonable intermediate state? I think that would fix most of the complaints I’ve heard recently about how things get handled.
Github does not have support for plugins (as far as I’m aware). Writing to STDOUT would give us something reasonably nice, especially when printed as a separate step. I think the current standard way to produce really nice outputs is to use the API to write a comment on the PR where everything can be formatted nicely. That will require quite a bit of additional handling though, partially due to security concerns.
This would require some amount of effort by someone with access to the current premerge infrastructure with how we handle container updates currently. We would also probably need to get all the dependencies installed and working on Windows, which could end up being a decent amount of work.
With one of the current goals being switching to Github Actions soon, I’m not sure any non-trivial time investment in getting this working with Buildkite is justified, especially when just appending the failure log at the end would still be quite useful now, in my opinion.