The idea
A service can fail before its CPU is full. Connections, memory, queue slots or a downstream quota may become the limiting resource first. Capacity planning starts by identifying which resource bounds the workload and how demand grows with concurrency.
Design choice
Measure resource use under representative payloads, including slow responses and large inputs. Set bounded concurrency and queue sizes. Reserve headroom for ordinary variation and recovery work, then define the signal that calls for scaling or rejecting excess demand.
Worked example
An export endpoint streams results but holds one database connection per request. A concurrency limit protects the shared connection pool so a burst of exports does not prevent ordinary interactive queries.
Before you ship
- Identify the first exhausted resource.
- Load-test a slow dependency.
- Publish the overload response and retry guidance.
Examples are illustrative. Adapt the checks to your own environment.