I should have mentioned this idea previously, but what about making the
script gracefully degrade in the absence of these tools? That way builders
won't regress but ones with the tools will just get better timing.
I've been running Hal Finkel's benchmarking mode patch along with perf
tool patch, and posted the results on http://146.169.44.158:55080/.
For the results I've collected until now, by defaulting to use median
instead of minimum, there isn't any same-run comparison false positive
result.
Cheers,
Yi Kong
I've been running Hal Finkel's benchmarking mode patch along with perf
tool patch, and posted the results on http://146.169.44.158:55080/.
Interesting.
For the results I've collected until now, by defaulting to use median
instead of minimum, there isn't any same-run comparison false positive
result.
I observe the same regularly for the full run. Even though the median may not be the theoretical best fit for all benchmarks, it seems to be filter most false positives.
Tobias
As all review comments were taken into account, I believe the patch is
ready to be committed. I plan on doing so sometime tomorrow unless any
objection is raised by then.
0001-Timeit-using-perf-on-Linux.patch (1.78 KB)
I think this is a great idea!
This won’t impact me though, since I don’t have any Linux testers.
CCing Daniel, in case he has an opinion.