> On the OVH box, I set the Linux CPU frequency governor to "performance", to make sure all requests get the fastest possible response
I'm not sure that'll meaningfully improve latency given how much more relevant other factors can be. Have you measured a real difference? If not, you might be burning CPU-cycles for nothing.
Some things felt noticeably faster to me, but that could of course be the placebo effect. More importantly, the CPU usage percentages reported by ps have fallen (those are actually large enough to measure in a few processes, because "100" means a single CPU thread and not the whole CPU). That implies that the CPU is in fact spending less time on some things.
> More importantly, the CPU usage percentages reported by ps have fallen
That's not a meaningful comparison when the CPU itself is running at a higher frequency. Lower frequency enables the CPU to waste less effort, e.g. while waiting on memory or I/O.
I don’t know if the same holds true in servers, but in mobile you want to race to idle. In other words, consistent high CPU usage at a low frequency is worse than spiking the frequency up to get CPU usage down to 0. I would imagine that holds too although it’s more complicated because a server usage model is different (perhaps a constant baseload to begin with changes the calculus).
Race to idle is good in a compute heavy workload (up to thermal limits), but I'm not sure it is when the CPU is mostly waiting for memory. Also IIRC, the point of using "performance" in the first place is that it keeps the CPU at its design frequency even when idle - so it will be less able to boost further when needed.
I'm not sure that'll meaningfully improve latency given how much more relevant other factors can be. Have you measured a real difference? If not, you might be burning CPU-cycles for nothing.