TL;DR: There’s now a working, documented flow for locally testing changes to the buildbot setup (new builders, or even broader llvm-zorg infra changes).
I’ve recently landed a series of changes which makes it substantially easier to test buildbot changes. Whether you’re adding a new builder config, changing an existing one, refactoring/improving Python logic in llvm-zorg, or deploying a new worker there is now a “local testing mode” flow that makes it easy to:
- Spin up a local buildbot instance (‘buildmaster’ in buildbot parlance)
- Connect the worker of your choice to it
- Review its output in a local version of the buildbot web UI at
http://localhost:8011.
The documentation should walk you through everything, including ssh port forwarding for remote machines. If you run into any issues, please let me know!
Relatedly, we now have some basic pre-commit checks via GitHub Actions for changes to the llvm-zorg repository (after llvm-zorg#303 lands this uses the ‘local testing mode’ to at least run buildbot checkconfig) which should help to avoid trivial issues in merged code.
If you’re wondering how you can help - at least one area of future improvement would be offering guidance on how to make use of an existing llvm-project repo checkout to avoid a from-scratch clone upon spinning up a test worker (and for the buildmaster’s git poller). PRs very welcome!
Thanks to @DavidSpickett and @omjavaid for help reviewing. Thanks to RISE for supporting the work that led to these improvements.