Chapter 0 left us somewhere uncomfortable: once an order spans several services, you cannot make it atomic, and you are guaranteed to sometimes be caught half-done. That's not a bug to fix — it's the ground you have to build on. This chapter is the solution, and — just as importantly — the handful of questions I had to answer before I could design a single table. Every one of those answers ends up as a constraint on the schema, so getting them right on paper first is the whole game.

A word on the setup, since it shapes the payment step: these are orders that were already placed — no customer is at a checkout. The batch fulfils them: reserve stock, record the payment, ship. And “payment” here means bookkeeping, not a card swipe — each order's amount is posted to a ledger account, one of the accounts in the company's chart of accounts, like the Cash account. So when I say authorize and capture, I mean staging that posting and then committing it to the ledger, entirely server-side. Nothing is charged to anyone in real time.

The idea: local transactions, plus a way to undo them

If you can't wrap one big transaction around four services, you do the opposite: you break the work into a sequence of local transactions, one per service — each of which is atomic, because it touches only one database. And for every forward step, you define a compensating action that semantically undoes it: a refund (not an un-charge), a release (not an un-reserve), a cancel.

When a step fails partway through, you don't roll back — there's nothing to roll back. You walk the completed steps in reverse and run each one's compensation. That's a saga.

✓ create order ✓ reserve stock ✗ settle payment fails — nothing to undo on it release stock cancel order no ROLLBACK — you run a compensating action for each completed step, in reverse: void → release → cancel

The failing step itself needs no undo — it failed, so it changed nothing durable. You compensate the ones that already committed, newest first. Compensation is a forward action that negates a previous one — which is the only kind of "undo" you get across services you don't control.

Orchestration, not choreography

There are two ways to run a saga. In choreography, each service reacts to the others' events — no one is in charge, and the workflow lives nowhere; you can't point at one place and see the flow. In orchestration, a central orchestrator owns the sequence: it calls step one, waits, calls step two, and on failure fires the compensations in reverse.

I chose orchestration, and it's the better teacher — the entire state machine is explicit and in one place. It's also the only style that can cleanly express things like "do these two steps at once, then wait for both," which choreography has no natural home for.

The orchestrator is a state machine — and the code is tiny

Stripped to its core, an orchestrator is a loop, a stack, and a try/catch:

async function runSaga(steps, ctx) {
  const done = [];                       // ← a STACK of completed steps
  try {
    for (const step of steps) {          // ← a LOOP, forward
      ctx[step.name] = await step.action(ctx);
      done.push(step);
    }
  } catch {
    for (const step of done.reverse())   // ← unwind BACKWARD
      await step.compensate(ctx);         // semantic undo
  }
}

Read it aloud: try to walk forward; if you trip, pop the stack backward and undo each thing you did. And the workflow itself is data — each step is a { action, compensate } pair — so you add a step without touching the engine. Everything else (persistence, retries, idempotency) is plumbing around this shape. But that plumbing is where the real questions live.

Question 1 — can the steps run in parallel?

My first instinct was that the four steps are a strict line. They're not. The orchestrator is really running a dependency graph, and sequential is just the case where every step needs the one before it. What decides parallelism is data dependencies: payment and inventory both need the order, but not each other — so they can run at once.

The catch: parallelism makes failure harder, not easier. If two branches run concurrently and one fails, you have to compensate the one that succeeded — coordinate a rollback across branches. So the rule I settled on:

  • Parallel + compensatable = hard — you need partial-failure rollback across branches.

  • Parallel + retryable = easy — both must eventually succeed, so a failed branch just retries; nothing to unwind.

Which is why, in the flow below, I keep the compensatable head sequential and only parallelize the retryable tail.

Question 2 — where does "undo" stop being possible?

This one reshaped everything. Compensation is a lie past a certain point. You can reverse a settlement, but you cannot un-ship a package — the goods have physically left. That point is the pivot: the one irreversible step. Before it, every step is compensatable; after it, every step is retryable — it must eventually succeed, so you retry it forever rather than roll back.

COMPENSATABLE — undo still works PIVOT RETRYABLE — no undo create · reserve · authorize ship capture · commit failure → compensate in reverse failure → retry forward, or escalate the pivot is the point of no return — once you've shipped, "undo" stops meaning anything

I learned this the hard way by walking through a bug: what if you've shipped, then commit the inventory, and the payment capture keeps failing — so you "compensate everything" and release the stock? Now your ledger says you have stock you already shipped. You corrupted the data by rolling back past the pivot. The fix is a rule, not code: never compensate past the pivot. After it, you retry forward, and if something genuinely can't complete, you escalate to a human — you never pretend the shipment didn't happen.

So the design job for each step became: decide which side of the pivot it's on. That single decision determines whether it gets a compensation or a retry — and it's baked into the schema as which tables carry a reversible status versus a forward-only one.

Question 3 — how do I not settle a payment twice on a retry?

The nastiest question, and it comes straight from Chapter 0's timeout: when a call fails, you can't tell whether the work happened and the reply was lost, or the request never arrived. So you retry — and risk doing it twice. The answer is idempotency: make an operation safe to repeat, so "retry to be safe" actually is safe.

client payment settle $50 · key = ORD-1:pay settles ✓, stores result under key 200 OK … lost in the network retry · SAME key = ORD-1:pay sees key → returns saved result does NOT settle again the deterministic key turns an unavoidable duplicate into a harmless no-op

The mechanism is a deterministic idempotency key — the same token on every retry (I use orderId:step). The service records what it did under that key; a repeat just returns the stored result without re-executing:

const prev = await store.get(key);   // "ORD-1:pay"
if (prev) return prev;                // seen it → return SAME result, don't re-settle
const result = await settle(...);
await store.set(key, result);         // record outcome, keyed by the token
return result;

Because the key is derived from the order and step — not from the attempt — a crash-and-replay and a duplicate delivery produce the same key, so both are harmless. This is what makes "just re-run the saga from where it died" a safe recovery strategy at all. And it's the reason the schema is full of unique constraints: the database is where the dedup is actually enforced.

Question 4 — how do the reports stay correct while orders are in flight?

A saga is eventually consistent — there's always a moment where the system is half-updated. So how does a stock or revenue report stay honest during that window? The answer is a principle: a report is a sum over rows filtered by status; it's correct if every row is in exactly one status bucket and the report counts the right ones.

That turns into two-phase modeling on both money and stock:

Held (in-flight)

Committed (final)

Report counts

Stock

reserved

sold

sellable = on-hand − reserved

Payment

authorized

captured

revenue = captured only

Reserving stock immediately drops "sellable" so two orders can't grab the last unit; authorizing money holds it without counting it as revenue. Only after the pivot do you commit — move reserved to sold, authorized to captured — and the reports flip to reflect reality. Anything mid-flight is simply excluded, and a periodic reconciliation job checks the invariants so you can promise correctness rather than hope for it.

The flow all of this produces

Put the answers together and the order flow designs itself:

create order reserve stock authorize ship PIVOT capture payment commit stock done Phase 1 · sequential · compensatable Phase 2 · parallel · retryable (join)

  • Phase 1 — sequential, compensatable: create the order, reserve stock, authorize payment. Anything here fails → compensate in reverse. Fail-fast, and the risky payment step happens while it's still reversible.

  • The pivot — ship. The irreversible act. Its success commits you forward; its failure (before dispatch) still lets you roll back.

  • Phase 2 — parallel, retryable: capture the payment and commit the stock at once, then join. No compensation to coordinate, so parallel is free — and it's the ideal place to run concurrently.

What this hands to the schema

None of the above is code yet — it's the set of decisions that constrain the code. And every one of them lands as a concrete requirement on the tables:

  • Idempotency → unique keys the database enforces (one order per id, one reservation per attempt-and-item).

  • The pivot → which tables carry a reversible status (reserved/released) versus a forward-only one.

  • Retries and reprocessing → the notion of a saga attempt, and "one active attempt per order" enforced by a partial-unique key.

  • Report correctness → separate held/committed columns on stock and payment, never one blended number.

  • Crash recovery → a durable saga log and a per-step log, so the orchestrator can resume mid-flight.

That's Chapter 2: turning these five answers into a set of MySQL tables that hold up under a crash, a duplicate message, a re-upload, and the cancel-then-resubmit case. The reason the schema comes after this chapter and not before is the whole lesson — in a distributed system, the data model is where your consistency decisions become real, so you make the decisions first and let the tables encode them.