Connected infrastructure

Budget a scarce resource

Editorial edition · 16 November 2026

The idea

A service can fail before its CPU is full. Connections, memory, queue slots or a downstream quota may become the limiting resource first. Capacity planning starts by identifying which resource bounds the workload and how demand grows with concurrency.

Design choice

Measure resource use under representative payloads, including slow responses and large inputs. Set bounded concurrency and queue sizes. Reserve headroom for ordinary variation and recovery work, then define the signal that calls for scaling or rejecting excess demand.

Worked example

An export endpoint streams results but holds one database connection per request. A concurrency limit protects the shared connection pool so a burst of exports does not prevent ordinary interactive queries.

Before you ship

  • Identify the first exhausted resource.
  • Load-test a slow dependency.
  • Publish the overload response and retry guidance.

Examples are illustrative. Adapt the checks to your own environment.

Further reading

HTTP Semantics, RFC 9110

Search the library