Benchmarking

April 6, 2025 ยท View on GitHub

We run various benchmarks to track performance regressions, measure simulated network traffic, etc. We try to ensure that the benchmarks are run in a repeatable, quiescent environment, but we may from time to time need to change the hardware on which the benchmarks are run, which will inevitably cause discontinuities in benchmark results. In the table below, we make a best-effort attempt to track the commits to main after which discontinuities may appear.

Breaking commitDescription
[82e1fce] (https://github.com/hackworthltd/primer/commit/82e1fcec836bac7a153014f721578a72140d4cde)Retired previous benchmark machine. For the time being, benchmarks will be run on a CI cluster, and may be a bit noisier than in the past.
d81d43e2Retired previous benchmark machine, a dedicated AMD Ryzen 5 3600 6-Core Processor host running only benchmark jobs, and moved subsequent benchmark jobs to a VM running other CI jobs, with an 8-core AMD Ryzen 9 5900HX with Radeon Graphics.
d38b9738Back to previous benchmark machine, as the mixed-use CI machine was not a reliable benchmarking host.
04867463Retired previous benchmark machine, and moved subsequent benchmark jobs to an 8-core AMD EPYC 7402P virtual machine.