Circuit Breakers

The claim under test

"The call either works or it throws." That sentence is the reason most service-to-service clients ship with no resilience configuration at all: if a failure surfaces as an exception, the existing catch block already handles it. The claim is not wrong so much as incomplete, and the missing case is the one that takes the service down.

Hang without a response is the third outcome, and it is not a rarer version of failure. It is a categorically different one. A thrown exception returns your thread. A hang keeps it. And a thread is not an abstraction: it is a finite, pre-allocated, shared resource that every endpoint in the process draws from.

The arithmetic of saturation

Put numbers on it, because the numbers are what make the failure feel inevitable rather than unlucky. A service with a 200-thread worker pool, 50 requests per second inbound, and one dependency that has started answering in 30 seconds instead of 30 milliseconds:

dependency      telemetry-service, alive and answering in 30 s
inbound         50 req/s, and every request touches it
worker pool     200 threads

each in-flight request holds exactly one thread
  threads consumed   50 per second
  pool drained in    200 / 50 = 4 seconds

Little's Law: concurrency = arrival rate × latency
  required           50 req/s × 30 s = 1500 threads
  available          200 threads
  shortfall          7.5×

the same service, one line of configuration different:
  timeout            1 s
  required           50 req/s × 1 s = 50 threads
  available          200 threads       → steady state, with headroom

Four seconds. That is the entire budget between "a dependency got slow" and "this service answers nothing". Nothing crashed, no exception was thrown, no error appeared in a log, and the health check, if it runs on the same pool, is now also queued behind 200 blocked workers, so the orchestrator will shortly restart a process whose only fault was politeness.

The last three lines of that block are the whole argument of this page in numeric form, and we will come back to them. A timeout does not make the dependency healthy. It makes the concurrency requirement bounded, which is the only property that keeps the pool solvent.

The cascade, mechanically

Saturation does not stay where it started. The service that ran out of threads is itself a dependency of something, and its callers are blocked on it for the same reason it is blocked on the thing beneath it. The fault travels upstream, against the direction of the call, one thread pool at a time.

Michael Nygard named the remedy, and Fowler is explicit about the attribution, worth getting right, because the pattern is routinely cited as Fowler's:

Chris Richardson's framing of the same idea states the problem as a question, which is a useful way to test whether you have the pattern or merely the annotation: "How to prevent a network or service failure from cascading to other services?" His answer, "A service client should invoke a remote service via a proxy that functions in a similar fashion to an electrical circuit breaker", puts the control on the caller, not the callee. The breaker protects you from your dependency. It does nothing for your dependency, and nothing for a dependency that has no breaker in front of it.

A timeout is the first line of defence

This is the part most write-ups skip, and skipping it is how teams ship a breaker that protects nothing. A circuit breaker is a statistical control. It reacts to outcomes. A call that never returns produces no outcome. It is not a success, it is not a failure, it is not even a slow call: the breaker computes the slow-call rate from completed calls, and a hung call has not completed.

You need two timeouts and they are not interchangeable. The connect timeout bounds how long you wait for a TCP handshake. It should be short, because a healthy peer answers in milliseconds and an unreachable one will not answer at all. The read or response timeout bounds how long you wait for the answer once connected, and it is the one that saves the thread pool. Above both, an application-level deadline (Resilience4j's TimeLimiter) bounds the whole operation including retries and redirects.

With a timeout in place, the breaker finally has something to work with: every call now terminates within a known bound, so the window fills with real outcomes and the failure rate becomes meaningful. That is the correct mental order: the timeout makes the failure observable; the breaker decides what to do about the pattern of failures. Neither substitutes for the other. A timeout alone still lets you spend 3 seconds per request discovering the same thing 50 times a second; a breaker alone never learns.

The three states

Resilience4j implements the pattern as a state machine, and the vocabulary is worth using precisely because each state has a different contract with the caller:

The transitions in the documentation's own words. Out of CLOSED: "When the failure rate is equal or greater than the threshold the CircuitBreaker transitions to open." And back:

Fowler describes the same third state in one sentence, "There is now a third state present - half open - meaning the circuit is ready to make a real call as trial to see if the problem is fixed", and its resolution in another: "Asked to call in the half-open state results in a trial call, which will either reset the breaker if successful or restart the timeout if not."

Tuning: the knobs and where to start

Two decisions dominate everything else: what the window measures and how much evidence is enough. Resilience4j offers two window shapes, "The count-based sliding window aggregrates the outcome of the last N calls" and "The time-based sliding window aggregrates the outcome of the calls of the last N seconds", and the difference matters most on quiet endpoints. A count-based window of 100 on an endpoint that receives four calls an hour is a window a day wide: a failure from yesterday morning still votes on whether to trip this afternoon.

The evidence threshold is minimumNumberOfCalls, and it exists for exactly one reason: a breaker must not trip on the first failure of a low-traffic endpoint. Its job, per the documentation, is to configure "the minimum number of calls which are required (per sliding window period) before the CircuitBreaker can calculate the error rate or slow call rate". Without it, one call, one failure, a 100% failure rate, and a breaker that opens on a single transient blip and denies service for the whole wait duration. With it set to 20, a single failure is 5% and the breaker correctly does nothing.

Slow calls are a separate trigger

A dependency answering every request successfully in four seconds has a zero percent error rate. An error-rate breaker will never open for it, and it destroys you exactly as described at the top of this page. This is why the slow-call rate is a distinct trigger with its own threshold rather than a refinement of the error rate: "The CircuitBreaker considers a call as slow when the call duration is greater than slowCallDurationThreshold", and when the percentage of slow calls crosses its own threshold the breaker transitions to open on that basis alone.

Set slowCallDurationThreshold below your TimeLimiter deadline. If they are equal, every slow call becomes a timeout before the breaker can count it as slow, and the slow-call trigger never fires. A threshold at roughly two-thirds of the deadline gives the breaker a band in which to notice degradation before it becomes failure.

Fallbacks, and the fallback that is a second outage

An open breaker fails in microseconds instead of seconds. That protects your thread pool, and on its own it changes a slow error into a fast one, which is a real win for stability and no win at all for the user. What the user experiences is decided by the fallback. There are three honest shapes:

1. Stale data, labelled as stale

   The last good answer, served from an in-process cache that the success path populates, with a marker the client can render: "telemetry as of 14:02". Best available outcome when the data is a snapshot of something that changes gradually. Requires the product decision to be made in advance: someone must agree that stale is better than absent, and for some data it is not.

2. Degraded response

   Return the screen without the part that failed: the mission view with an empty telemetry panel and an explicit "unavailable" state, rather than a 500 for the whole page. This is usually the highest-value option and the one that requires the most work, because the response contract has to admit partial answers from the beginning.

3. Fail fast, and say so

   A 503 with Retry-After, returned immediately. Unglamorous, and correct when there is no meaningful degraded answer: a write path, an authorisation decision, anything where a wrong answer is worse than no answer. Failing fast is still a fallback: it converts an indefinite hang into a bounded, honest signal that a caller can act on.

The test is mechanical: if the fallback can block, it is not a fallback. A remote cache is a network call. A database read is a network call. A synchronous log shipment is a network call. If you genuinely need a remote secondary, it needs its own breaker, its own timeout and its own bulkhead, at which point you are running two protected dependencies, and the fallback of the fallback still has to be local.

The two patterns that travel with it

A breaker alone is a partial control. It acts per dependency, after enough evidence has accumulated, and it says nothing about how much of your process a single dependency may own in the meantime. Two companions close those gaps, and one of them is probably already in your codebase making things worse.

Bulkheads: isolating the pools

The bulkhead takes its name from a ship's compartments: a hull breach floods one compartment instead of the vessel. Applied to a service, it caps how many threads any one dependency may hold, so a dependency that goes slow can consume its quota and no more. Where a breaker responds to a pattern of failures, a bulkhead enforces a hard limit from the first call, including during the window before the breaker has enough evidence to act, which is exactly when saturation happens.

The semaphore variant is the one to reach for first: maxConcurrentCalls permits handed out on the calling thread, with maxWaitDuration set to zero so a saturated bulkhead rejects immediately rather than parking a thread waiting for a permit, because queueing for a permit is a smaller version of the problem you are solving. Size the quotas so that the sum across all dependencies is comfortably under the worker pool. If five dependencies each get 20 permits out of a 200-thread pool, 100 threads are always available for work that calls nothing, and the bottom-left box in the diagram above never turns red.

Retries: the amplifier you already have

Retries are usually added long before breakers, by someone reasonable, for a good reason, and they are the fastest way to convert a degraded dependency into a dead one. The arithmetic is brutally simple: a service at its limit receiving 10,000 requests per second starts failing 1% of them, each failure is retried, and the SRE book traces where that goes: "Those 100 failed QPS are retried in MakeRequest every 1,000 ms, and probably succeed. But the retries are themselves adding to the requests sent to the backend, which now receives 10,200 QPS." The feedback loop is positive, which is the definition of a cascading failure quoted earlier.

Worse, retries synchronise. Every caller that failed at the same instant, and they all failed at the same instant because they all called the same sick dependency, waits the same interval and returns together. Backoff spreads the waves out but does not desynchronise them, which is the observation behind the standard fix:

The SRE book gives the same instruction as a rule, "Always use randomized exponential backoff when scheduling retries", with the reason attached: "If retries aren't randomly distributed over the retry window, a small perturbation can cause retry ripples to schedule at the same time." It also names the ceiling most implementations lack: "Consider having a server-wide retry budget. For example, only allow 60 retries per minute in a process, and if the retry budget is exceeded, don't retry; just fail the request."

The problem this would solve

That last point deserves the arithmetic, because it is the number that turns an architecture diagram into a service-level objective. Assume all five dependencies are required to answer the request and that their failures are independent, the friendliest possible assumptions, and both of them optimistic:

the busiest service calls five others synchronously while handling a request

each dependency at   99.9 %   ( 43.2 min of downtime per month )

all five required, failures independent:
  0.999 ^ 5  =  0.999 × 0.999 × 0.999 × 0.999 × 0.999
             =  0.995010
             =  99.501 %

so before this service's own bugs, its own GC and its own deploys:
  ceiling              99.501 %
  downtime             3 h 36 min per month   ( 43.7 h per year )
  cost of the fan-out  5× the downtime of any single dependency

a breaker does not raise this ceiling.
it converts "hung for 30 s" into "degraded in 3 ms".
the fallback is the only thing that raises effective availability.

Three consequences follow, and only the third is the breaker's job. First, the ceiling is structural: no amount of care inside this service buys back the 0.499% its dependencies spend. Second, the honest fixes are architectural: make a dependency asynchronous, make it optional, or cache its answer. Each removes a factor from the product. Third, for the calls that must stay synchronous and required, the breaker plus a fallback changes what "unavailable" means: not a hung request and a drained pool, but a fast, partial, labelled answer. The multiplication still happens. What it multiplies is no longer an outage.

Doing it properly

Three tabs, in the order the change happens. The first is what exists; the second is the protection and the configuration that drives it; the third is the fallback, which is where most implementations quietly reintroduce the problem.

package estate.mission.service;

import java.util.List;

import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Service;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;

/**
 * One of sixteen call edges shaped exactly like this. The url attribute pins
 * the client to a single address, so the name never reaches a discovery
 * client. There is no breaker and no bulkhead.
 *
 * Be precise about the timeout, because the sloppy version of this claim is
 * wrong: Spring Cloud OpenFeign DOES register a default Request.Options,
 * currently 10s connect and 60s read. So the call is bounded - but bounded at
 * 60 seconds, which for a user-facing request is indistinguishable from
 * hanging, and is far longer than any pool can absorb under load. The defect
 * is not "unbounded", it is "bounded by an accident rather than a decision".
 */
@FeignClient(name = "telemetry-service", url = "${telemetry.base-url}")
interface TelemetryClient {

    @GetMapping("/telemetry/{areaId}")
    List<TelemetrySample> latestFor(@PathVariable String areaId);
}

/**
 * The busiest service in the estate. Five synchronous dependencies on the
 * request path, resolved one after another on the worker thread that is
 * serving the user. Its availability is the product of theirs, and its
 * thread pool is shared with every endpoint it exposes, including the ones
 * that call none of these clients.
 */
@Service
class MissionAssembler {

    private final AreaClient areas;
    private final CrewClient crew;
    private final TelemetryClient telemetry;
    private final GeofenceClient geofences;
    private final AlertClient alerts;

    MissionAssembler(AreaClient areas, CrewClient crew, TelemetryClient telemetry,
                     GeofenceClient geofences, AlertClient alerts) {
        this.areas = areas;
        this.crew = crew;
        this.telemetry = telemetry;
        this.geofences = geofences;
        this.alerts = alerts;
    }

    /**
     * Five blocking calls on one thread. If any single dependency starts
     * answering in 30 s instead of 30 ms, this thread is held for 30 s, and
     * so is every other thread that reaches this method. At 50 req/s the
     * 200-thread pool is gone four seconds later.
     */
    MissionView assemble(String missionId, String areaId) {
        var area       = areas.byId(areaId);
        var onDuty     = crew.onDutyIn(areaId);
        var samples    = telemetry.latestFor(areaId);
        var fences     = geofences.forArea(areaId);
        var openAlerts = alerts.openFor(missionId);

        return MissionView.of(missionId, area, onDuty, samples, fences, openAlerts);
    }
}

package estate.mission.service;

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;

import org.springframework.stereotype.Service;

import io.github.resilience4j.bulkhead.annotation.Bulkhead;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import io.github.resilience4j.retry.annotation.Retry;
import io.github.resilience4j.timelimiter.annotation.TimeLimiter;

/**
 * Resilience4j documents the default aspect order as
 *
 *   Retry ( CircuitBreaker ( RateLimiter ( TimeLimiter ( Bulkhead ( Function ) ) ) ) )
 *
 * Note that the ORDER OF THE ANNOTATIONS BELOW DOES NOT DECIDE THIS. The
 * nesting is governed by the *AspectOrder properties (retryAspectOrder,
 * circuitBreakerAspectOrder, timeLimiterAspectOrder, bulkheadAspectOrder),
 * where a higher value means higher priority and therefore an outer position.
 * The annotations are written in the default order here so that reading the
 * class matches reading the runtime; they are not what produces it.
 *
 * Retry outermost: every attempt passes through the breaker, so once the
 * breaker is OPEN an attempt is rejected in microseconds instead of putting
 * another call onto a dependency that is already sick.
 *
 * TimeLimiter INSIDE the breaker: a call that blows its deadline throws
 * TimeoutException inside the breaker, so the breaker records it. Move the
 * limiter outside and the breaker never observes the failure that matters
 * most - the slow one - which is how a breaker ends up permanently CLOSED
 * through an outage.
 *
 * Bulkhead innermost: a permit is spent only on a call that is actually
 * going out. A call the breaker already rejected must not consume
 * concurrency it never used.
 *
 * Invert Retry and CircuitBreaker and the breaker sees one outcome per retry
 * group: it records a single failure where three network calls happened, so
 * it trips roughly three times slower while the retries pile load onto the
 * dependency that is already failing.
 */
@Service
class TelemetryGateway {

    private final TelemetryClient telemetry;
    private final TelemetryCache cache;
    private final Executor io;

    TelemetryGateway(TelemetryClient telemetry, TelemetryCache cache, Executor io) {
        this.telemetry = telemetry;
        this.cache = cache;
        this.io = io;
    }

    /**
     * CompletableFuture is not decoration. @TimeLimiter can only abandon a
     * call it did not make on the caller's thread; a synchronous signature
     * here would silently disable the deadline. The executor is bounded and
     * dedicated - never the common ForkJoinPool, which is sized for CPU work
     * and starves under blocking I/O.
     *
     * lastKnownGood is on the next tab. It touches no network.
     */
    @Retry(name = "telemetry")
    @CircuitBreaker(name = "telemetry", fallbackMethod = "lastKnownGood")
    @TimeLimiter(name = "telemetry")
    @Bulkhead(name = "telemetry")
    public CompletableFuture<List<TelemetrySample>> latestFor(String areaId) {
        return CompletableFuture.supplyAsync(() -> {
            var samples = telemetry.latestFor(areaId);
            cache.put(areaId, samples);   // the success path feeds the fallback
            return samples;
        }, io);
    }
}

resilience4j:

  circuitbreaker:
    configs:
      default:
        # TIME_BASED, because a count-based window on a quiet endpoint can
        # span hours of wall clock: a failure from this morning would still
        # be voting on whether to trip this afternoon.
        slidingWindowType: TIME_BASED
        slidingWindowSize: 60
        # A breaker must not trip on the FIRST failure of a low-traffic
        # endpoint. Below this many calls no rate is computed at all.
        minimumNumberOfCalls: 20
        failureRateThreshold: 50
        # A SECOND, INDEPENDENT TRIGGER. A dependency that answers every
        # call successfully in 4 s has a 0% error rate and is still the
        # thing that drains the pool. Kept below timeoutDuration so a call
        # can be counted slow before it is counted failed.
        slowCallDurationThreshold: 2s
        slowCallRateThreshold: 50
        waitDurationInOpenState: 10s
        # The trial is a probe, not a load test. Unbounded trial traffic
        # re-kills a cold dependency and turns recovery into an oscillator.
        permittedNumberOfCallsInHalfOpenState: 3
        # false: the breaker half-opens on the next call after the wait has
        # elapsed, not on a timer. Set true only if the transition must
        # happen without any traffic to drive it.
        automaticTransitionFromOpenToHalfOpenEnabled: false
        # A 404 is an answer, not a dependency failure. Counting client
        # errors trips the breaker on your own bug and hides it.
        ignoreExceptions:
          - estate.mission.error.ResourceNotFoundException
        registerHealthIndicator: true
    instances:
      telemetry:
        baseConfig: default

  timelimiter:
    configs:
      default:
        # THE FIRST LINE OF DEFENCE. Without this the breaker can only react
        # to calls that eventually return, and a hung call never returns.
        timeoutDuration: 3s
        cancelRunningFuture: true
    instances:
      telemetry:
        baseConfig: default

  bulkhead:
    configs:
      default:
        # A hard ceiling on how much of the worker pool one dependency may
        # own, enforced from the first call rather than after 20 of them.
        maxConcurrentCalls: 20
        # Do not queue for a permit. Waiting for one is a smaller version of
        # the saturation this is here to prevent.
        maxWaitDuration: 0
    instances:
      telemetry:
        baseConfig: default

  retry:
    configs:
      default:
        maxAttempts: 3
        waitDuration: 200ms
        enableExponentialBackoff: true
        exponentialBackoffMultiplier: 2
        exponentialMaxWaitDuration: 2s
        # JITTER. Without it, every caller that failed at t=0 retries
        # together at t=200ms and again at t=400ms - a synchronised second
        # wave onto a dependency that has not recovered yet.
        enableRandomizedWait: true
        randomizedWaitFactor: 0.5
        retryExceptions:
          - java.io.IOException
          - java.util.concurrent.TimeoutException
        ignoreExceptions:
          # Retrying against an OPEN breaker burns attempts for no purpose:
          # the answer is already known. Give up on the first rejection.
          - io.github.resilience4j.circuitbreaker.CallNotPermittedException
          # Never blind-retry a non-idempotent write.
          - estate.mission.error.NonIdempotentOperationException
    instances:
      telemetry:
        baseConfig: default

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,circuitbreakers,circuitbreakerevents
  health:
    circuitbreakers:
      enabled: true

package estate.mission.service;

import java.time.Instant;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;

import io.github.resilience4j.circuitbreaker.CallNotPermittedException;

/**
 * Resilience4j resolves a fallback by signature: identical arguments,
 * identical return type, one extra Throwable parameter. The most specific
 * match wins, so an overload typed to CallNotPermittedException lets an OPEN
 * breaker answer differently from a single failed call - useful, because
 * "the dependency is known bad" and "one call did not work" are different
 * facts to put in front of a user.
 */
class TelemetryFallbacks {

    private final TelemetryCache cache;

    TelemetryFallbacks(TelemetryCache cache) {
        this.cache = cache;
    }

    /**
     * The breaker is OPEN. We already know the dependency is unavailable, so
     * there is nothing to attempt and nothing to wait for.
     */
    CompletableFuture<List<TelemetrySample>> lastKnownGood(String areaId,
                                                            CallNotPermittedException open) {
        return CompletableFuture.completedFuture(degrade(areaId));
    }

    /** Any other failure: timeout, IO error, rejected bulkhead permit. */
    CompletableFuture<List<TelemetrySample>> lastKnownGood(String areaId, Throwable cause) {
        return CompletableFuture.completedFuture(degrade(areaId));
    }

    /**
     * IN-PROCESS ONLY. One map lookup. No socket, no disk, no lock held
     * across I/O, and no dependency on anything outside this JVM. Its worst
     * case is measured in microseconds, which is the entire point: this code
     * runs when the thread budget is already spent.
     */
    private List<TelemetrySample> degrade(String areaId) {
        var cached = cache.peek(areaId);
        if (cached == null) {
            // No stale answer to give. Fail fast and say so, rather than
            // inventing data: the caller renders an "unavailable" panel and
            // the rest of the screen still works.
            return List.of();
        }
        // Stale, and labelled as stale. The client shows "as of 14:02"
        // instead of presenting an old reading as a current one.
        return cached.markStale();
    }

    // private List<TelemetrySample> degrade(String areaId) {
    //     return backupRegionClient.latestFor(areaId);   // NEVER
    // }
    // A fallback that makes a network call is a second unbounded dependency
    // on the failure path, reached at the moment there are no threads left,
    // and very likely unhealthy for the same reason as the primary.
}

/**
 * The cache the success path fills. Bounded and in-memory by construction -
 * a remote cache here would put a network call back on the failure path and
 * undo the whole exercise.
 */
class TelemetryCache {

    private final Map<String, StaleAware> entries = new ConcurrentHashMap<>();

    void put(String areaId, List<TelemetrySample> samples) {
        entries.put(areaId, new StaleAware(samples, Instant.now()));
    }

    StaleAware peek(String areaId) {
        return entries.get(areaId);
    }
}

record StaleAware(List<TelemetrySample> samples, Instant asOf) {

    List<TelemetrySample> markStale() {
        return samples.stream().map(s -> s.withAsOf(asOf)).toList();
    }
}

Before you ship it

Adding the annotation satisfies none of this. Each line below is a thing to verify, and the last one is the only one that produces evidence rather than intent.

• Every outbound call has an explicit connect and read timeout, set deliberately rather than inherited from the library default.
• The TimeLimiter deadline is at or below the caller's own deadline, and the transport timeout is at or below the TimeLimiter, so a cancelled future does not leave a socket read running behind it.
• slowCallDurationThreshold is set and is below the deadline, so degradation is detectable before it becomes timeouts.
• minimumNumberOfCalls is sized for the traffic this endpoint actually receives, not for the traffic the busiest endpoint receives.
• ignoreExceptions excludes domain and client errors, so a 404 cannot trip the breaker.
• The fallback makes no network call, acquires no lock across I/O, and is covered by a test that asserts the degraded shape.
• A bulkhead caps each dependency, and the sum of the caps is comfortably below the worker pool size, leaving headroom for endpoints that call nothing.
• Retries are bounded, use exponential backoff with jitter, ignore CallNotPermittedException, and are absent from every non-idempotent write that has no idempotency key.
• Aspect order is the documented default, or the *AspectOrder properties are set with a written reason.
• Breaker state is on a dashboard, transitions to OPEN are logged and alerted, and the fallback rate is measured separately.
• You have watched it trip. Inject a 30-second delay into one dependency in a non-production environment and confirm the breaker opens, the fallback serves, the unrelated endpoints keep answering, and the breaker closes again afterwards. A breaker that has never been observed to trip is a configuration, not a control.

Canonical sources

• Martin Fowler: CircuitBreaker · the definition, the third state, and the logging requirement. Fowler credits Michael Nygard's Release It! with popularising the pattern, so it is not his.
• Resilience4j: CircuitBreaker · the state machine, both sliding-window types, and every property name used in the configuration above.
• Resilience4j: Bulkhead · the semaphore and fixed-thread-pool implementations, and what each limits.
• Resilience4j: Spring Boot 3 getting started · the documented default aspect order and the *AspectOrder properties that govern it.
• Chris Richardson: Circuit Breaker · the problem stated as a question, and the solution placed on the caller.
• Google SRE Book: Addressing Cascading Failures · the definition of a cascading failure, thread starvation, retry amplification and the retry budget.
• Marc Brooker: Exponential Backoff And Jitter · why backoff alone still clusters, and why jitter is the fix. Cited in place of the AWS Builders' Library article on the same subject, whose URL now redirects to a page that serves no readable text.