What they're testing
Whether you think about the aggregate effect of many clients, or only about one.
The short answer~30 seconds
Exponential backoff so you don't hammer, random JITTER so thousands of clients don't retry in unison, and a hard budget so you don't retry forever. Jitter is the most-skipped and most important part: without it, every client that failed together retries together, producing synchronised waves against a service trying to recover — precisely when it needs less load.
The dependency starts failing. All 400 clients receive the error at the same instant — that shared starting point is the whole problem.
The long answer
Only retry things that might be transient: timeouts, connection errors, 429, 503. Retrying a 400 or 422 is pointless because the request is still wrong, and it only burns your budget. With 5xx be more careful: if the operation isn't idempotent, a retry can duplicate — the server may have completed the work and failed on the way back. Which is why retries and idempotency keys always travel together.
The structural trap is nested retries. If layer A retries three times, layer B inside it also retries three times and so does C, one user request becomes 27 calls to the bottom service. It's a very common way to build an attack on yourself. The pragmatic rule: retry at ONE layer, usually the one closest to the failure, and have the layers above simply propagate.
A circuit breaker is the necessary extra layer when failures persist. After N consecutive failures the circuit opens and further calls fail IMMEDIATELY without leaving the process — you stop wasting time on timeouts and stop adding load to a dying service. After a cool-off it goes half-open and lets a few probes through. The core idea: when failure is near-certain, failing fast beats failing slowly.
// Full jitter (khuyến nghị của AWS): chọn ngẫu nhiên trong [0, backoff]
async function retry<T>(fn: () => Promise<T>, max = 4): Promise<T> {
for (let attempt = 0; ; attempt++) {
try { return await fn(); }
catch (err) {
if (attempt >= max || !isTransient(err)) throw err;
const cap = Math.min(30_000, 200 * 2 ** attempt);
await sleep(Math.random() * cap); // <- jitter, không phải cap cố định
}
}
}What they'll ask next
?What timeout do you set?
Start from that service's p99 plus headroom, not an arbitrary round number. And the total across all retries must be less than your caller's timeout — otherwise the caller gives up while you're still retrying, and the work is entirely wasted.
?What's a retry budget?
A cap on the AGGREGATE retry rate across the client, e.g. retries may not exceed 10% of normal traffic. It prevents the state where every request is on its third attempt, which a per-request limit can't.
These lose points
- Retrying immediately with no backoff. The fastest way to turn a blip into an incident.
- Exponential backoff without jitter. You still get synchronised waves, just further apart.