What they're testing
Whether you distinguish CPU-bound from I/O-bound work — the basis of every concurrency decision.
The short answer~30 seconds
For CPU-bound work: the core count, because adding more helps nothing when no core is free. For I/O-bound: cores × (1 + wait time / compute time) — if 90% is spent waiting on the network, that's an order of magnitude above the core count. But don't stop at the formula: the real limit is usually DOWNSTREAM (the database pool, a third-party quota), so computing 200 threads when the database accepts 20 connections just makes the queue longer.
The long answer
Worth stating: the queue is a parameter too, and usually the more important one. A ten-thread pool with an unbounded queue never rejects work — it accumulates until memory runs out, with latency rising without limit while everything looks "fine". A BOUNDED queue with an explicit rejection policy (return 503, or push back on the caller) makes the system fail visibly, which is what you want.
The classic trap is sharing one pool between blocking and non-blocking work. A task calling an external API occupies a thread for the whole network wait; enough of them and the pool is exhausted and even fast tasks can't run. The remedy is a bulkhead: a separate pool per dependency, so one slow service can't take the rest down with it.
On a modern JVM the question is changing shape: virtual threads make blocking cheap enough that pools are largely unnecessary for I/O work. But the principle survives — you still cap concurrency where a real limit exists, using a semaphore instead of a pool size. Saying that shows you understand the problem rather than the formula.
What they'll ask next
?Which metric tells you the pool is mis-sized?
Queue depth and time spent waiting in it. An always-empty queue with unsaturated CPU means you can go higher; a long queue with saturated CPU means more threads just slow everything uniformly. Busy-thread count alone tells you nothing.
These lose points
- Using
Executors.newCachedThreadPool()for uncontrolled load. It creates threads without bound and will kill the JVM in a spike.
Sources
- Brian Goetz, Java Concurrency in Practice, ch. 8 — pool sizing and queues
- Java SE API — `ThreadPoolExecutor` (rejection policies)