Performance tracking and benchmarking infrastructure BoF

As preparation for the performance tracking and benchmarking infrastructure BoF next week at the dev meeting, Chad, Chris, Tobias, Renato and myself have started brainstorming and trying to somewhat organize the topics we feel should be discussed.

We’ve got a google docs document with an outline of what we think should be discussed. We very much welcome comments, suggestions and feedback on it before the BoF, so that we can make the best possible use of the allocated BoF time next week.

At the moment, the top-level topics we’d like to discuss are:

  • Producing low-noise performance numbers

  • Easily interpreting performance numbers

  • Acting on Regressions

  • Enhanced data collection

  • Getting a stable and high-quality warning system

  • Making a larger part of the community care

  • Guidelines for setting up a performance tracking bot

  • Making performance tracking server more stable and easier to administer

More details are in the google docs document, accessible through the link below. Feel free to add comments directly to the google docs

document, or by replying to this email.

https://docs.google.com/document/d/19gqxdRoZ4tokgUQy4LdpI7FztYRLgQNRxYjMZe1LA9E/edit?usp=sharing

Thanks,

Kristof