What they're testing
Whether you think about performance statistically, or just read one number off a dashboard.
The short answer~30 seconds
Latency distributions are right-skewed: most requests are fast, a few are very slow. The average is dragged toward the fast crowd, so it doesn't describe the worst experience — and the worst experience is what makes people leave. p99 means 99% of requests are faster than that number. It sounds like "only 1% affected", but if a page makes 40 API calls, the chance at least one lands in the tail is roughly 33% — so a third of page loads hit p99.
The long answer
That arithmetic is worth spelling out because it changes priorities: the chance of avoiding the tail is 0.99⁴⁰ ≈ 0.67. So "p99 affects 1% of users" is wrong in any system with several dependent calls, and worse in microservices where each call fans out further. This is what Google named the tail at scale.
A second technical trap: percentiles don't average. You can't take the p99 from ten servers and average them into a system p99 — that number means nothing. Correctly, you merge histograms and compute the percentile from the merge, which is why modern metric systems store histograms rather than pre-computed percentiles.
On what to track, I use p50 for the typical experience, p95 or p99 for the SLO, and always alongside throughput. A beautiful p99 at 3am says nothing; peak-hour p99 is the number to defend. And if p99 suddenly IMPROVES during an incident, it's usually because the slow requests are now erroring and dropping out of the measurement — a very common misreading.
What they'll ask next
?How should you set an SLO?
From what users actually need, not from what you currently achieve. The error budget is the complement: a 99.9% SLO allows about 43 minutes a month out of spec. That budget is the negotiating tool between release speed and stability, and that's its purpose.
These lose points
- Reporting performance as a single average. Arithmetically true and operationally useless.
- Averaging percentiles. The most common statistical error in performance dashboards.