Bulkhead

The name comes from ships. The hull of a ship is divided into watertight compartments by walls called bulkheads. If the hull is damaged, water fills one compartment - and stops there. The ship stays afloat. Without the walls, one hole sinks the whole ship.

In software, the "water" is a dependency that becomes slow or broken, and the compartments are your service's resources: connections, threads, worker slots, memory. The Bulkhead pattern gives each dependency, or each kind of work, its own limited share of those resources. When one dependency fails, it can use up only its own share. Everything else keeps working.

This lesson follows Order Service, which calls two other services: Payment (to charge an order) and Inventory (to check stock). You will see Payment become slow and drag Inventory down with it - and then build the walls that stop it.

Every outgoing call needs a resource while it waits for an answer: a connection, a thread, a slot. These come from a pool of limited size. Most services, by default, use one pool for all their outgoing calls.

Step through the diagram. Payment becomes slow - 3 seconds per call. The orders that need Payment take every connection in the pool and hold them for 3 seconds. Then stock checks arrive. Inventory is completely healthy, but there is no free connection to reach it. In our real run, a stock check that normally takes 20 ms took about 6 seconds. To the user, Inventory looks broken. It is not - Payment is.

With bulkheads, Order Service keeps a separate pool for each dependency: 5 connections for Payment, 5 for Inventory. Payment can still use up its own 5 - but it cannot touch Inventory's. Stock checks stayed at 30 - 53 ms while Payment was slow.

There is a second half to the pattern. A full compartment should not collect an endless queue of waiting work: with only separate pools, the last orders waited up to 12 seconds. So the Payment bulkhead also limits its waiting area - here, 5 running and 5 waiting. Anything beyond that is refused at once, in 16 ms, with a clear "busy, try again" answer. Failing fast is better than making a user wait 12 seconds for a failure.

A bulkhead is any limit that keeps one kind of work from using resources another kind needs. In the lab you build two kinds: separate connection pools, and a concurrency limit with a small queue. The same idea appears at every level of a system.

A useful rule, called Little's Law: the number of calls in progress at the same time equals how many calls arrive per second multiplied by how long each call takes. If Payment receives 20 calls per second and each takes 0.1 seconds, about 2 calls are in progress at any moment on a normal day.

Set the limit a few times above that - say 5 or 6 - so normal bursts fit, and keep the waiting area small. If the limit is too small, good requests are refused on a normal day. If it is too big, a slow dependency can still hold a large share of your resources before the wall stops it. Watch two numbers in production: how often the bulkhead is full, and how many requests it refuses.

Bulkheads are usually combined with the patterns from the circuit breaker lesson, because each one limits something different. A timeout limits how long one call may wait. A bulkhead limits how many calls may wait at the same time. A circuit breaker stops making calls at all when most of them fail.

In the circuit breaker lab, a burst of 20 orders arriving at the same moment all reached Payment, because the breaker had nothing to judge yet. A bulkhead is exactly what limits such a burst: it does not need to wait for failures - it only counts how many calls are in progress.

Bulkheads are not free. Resources split into compartments cannot be shared: if Inventory is idle and Payment is busy, Payment still cannot borrow Inventory's connections. You need to choose and watch the sizes. And refusing requests is only better than waiting if the caller handles the refusal well - a clear message, a retry later, a fallback.

You probably do not need bulkheads for a service with a single dependency, or for internal tools where a slow afternoon is acceptable. You do need them when one service calls several others and some of those calls matter more than others - checkout versus recommendations, login versus reports.

What is the Bulkhead pattern? Isolating resources - pools, threads, queues, instances - per dependency or per workload, so a failure in one part can use up only that part's resources and the rest of the system keeps working.

How is a bulkhead different from a circuit breaker? A circuit breaker decides whether to call a dependency, based on its recent failures. A bulkhead limits how many calls to it can be in progress, regardless of failures - so it also works against sudden bursts.

How do you size a bulkhead? Start from normal concurrency (Little's Law: arrival rate x latency), add headroom for bursts, keep the queue small, and adjust using the measured rate of full and rejected requests.

What happens to requests when the bulkhead is full? They are rejected immediately, and the caller should return a clear "busy" response, use a fallback, or retry later with backoff - not wait without limit.

You need Node.js 18 or newer. Install Express for the services and cockatiel, a resilience library for Node.js that includes a bulkhead. The project has three services: Payment on port 3002, Inventory on port 3003, and Order Service on port 3001, which calls both.

Payment Service answers in 20 ms, or in 3 seconds when you switch it to "slow". It also records the largest number of payment calls it handled at the same time - that number shows whether a bulkhead is working. Inventory Service always answers in about 20 ms.

Order Service will make its calls through http-call.js. It uses Node's built-in http module because that lets you choose the connection pool for each request: the agent option. An Agent is Node's connection pool, and choosing which agent each call uses is exactly how you build the walls.

The first Order Service uses one Agent for every outgoing call, with at most 10 connections in total (maxTotalSockets). load.js sends 20 orders at once and, 100 ms later, 10 stock checks, then prints how long each kind took.

With Payment healthy, everything is fast: stock checks 27 - 29 ms. With Payment slow, the 20 orders filled the pool, and the stock checks - which only need the healthy Inventory - took about 6 seconds each. Payment's own counter shows 10 calls at once: it had the whole pool.

Many services turn on keepAlive, which keeps connections open after a request so the next one is faster. We tried the shared pool with keepAlive: true and Payment completely healthy. The orders finished in about 60 ms - and then the stock checks took about 6 seconds anyway.

The reason: the 10 kept-open connections to Payment were idle, but they still counted towards the shared limit of 10. Inventory had to wait until they were closed, about 5 seconds later. Nothing was slow and nothing was broken - the shared pool alone made a healthy call 300 times slower. Separate pools do not have this problem, because Payment's idle connections never count against Inventory's.

The bulkhead version has two Agents: paymentPool and inventoryPool, each limited to 5 connections (maxSockets). Each route uses its own pool. That is the whole change.

With Payment slow, the stock checks stayed at 38 - 58 ms. But look at the orders: the slowest one waited 12 seconds. 20 slow payments shared 5 connections, so they went through in four rounds of 3 seconds. The wall protected Inventory, but Payment's compartment collected a long queue.

cockatiel's bulkhead(5, 5) allows 5 calls to run and 5 more to wait. When both are full, execute() throws BulkheadRejectedError at once, and the route answers HTTP 503 with a clear message. Inventory keeps its own pool as before.

With Payment slow: stock checks 30 - 53 ms. 10 orders were served (5 after 3 seconds, 5 after 6), and 10 were refused in 16 ms instead of waiting up to 12 seconds. Payment never had more than 5 calls at once. While it was busy, /admin/bulkhead showed 5 running and 0 free waiting places.

check.js makes Payment slow, sends 20 orders and 10 stock checks, and checks four things: stock checks stay under 300 ms, Payment never handles more than 5 calls at once, some orders are refused, and refused orders are answered in under 200 ms.

Against order-service-limited.js all four checks passed. Against order-service-shared.js three failed - including a stock check that took 6.2 seconds.