The Distributed Saga Pattern: Transactions in Microservices

The Distributed Saga Pattern: Transactions in Microservices

In a monolithic architecture backed by a relational database, business transactions are trivial.

If a user books a flight and a hotel, you wrap both operations in a single BEGIN ... COMMIT block. If the flight books successfully but the hotel fails, the database automatically rolls back the flight booking. You have perfect, atomic consistency guaranteed by the ACID properties of the database.

When you decompose your monolith into microservices, you destroy this capability. The Flight Service has its own database, and the Hotel Service has its own database. You cannot wrap a network call in a database transaction.

If the Flight Service successfully books the flight, and then the HTTP call to the Hotel Service times out, what do you do? You now have an incredibly dangerous, partially completed business transaction. The customer paid for a flight but has no hotel.

You cannot use Two-Phase Commit (2PC) in a modern microservice architecture because it relies on synchronous locking across the network, which destroys availability and performance.

The industry standard solution for distributed transactions is the Saga Pattern.

What is a Saga?

A Saga is a sequence of local transactions. Each local transaction updates the database of a single microservice and publishes a message/event to trigger the next local transaction in the sequence.

If a local transaction fails (e.g., the hotel is fully booked), the Saga executes a series of Compensating Transactions that undo the changes made by the preceding local transactions.

The Forward Recovery vs. Backward Recovery

Sagas operate under the assumption of Eventual Consistency.

• Forward Recovery: If a service is temporarily down, the Saga simply keeps retrying the local transaction until it succeeds. This works when failure is transient (e.g., a network blip).
• Backward Recovery (Compensation): If a service permanently rejects the transaction (e.g., "Insufficient Funds" or "Out of Stock"), the Saga must pivot and execute compensating transactions backwards through the chain to restore the system to its original state.

Crucial Note: A compensating transaction is not a database rollback. The original transaction was already committed to the database. A compensating transaction is a brand new transaction that semantically reverses the original action (e.g., if the original action was ChargeCreditCard($100), the compensation is RefundCreditCard($100)).

Choreography vs. Orchestration

There are two primary ways to coordinate a Saga: Choreography and Orchestration.

1. Choreography (Event-Driven)

In a Choreography-based Saga, there is no central controller. Services publish domain events to a message broker (like Kafka), and other services listen and react.

The Flow:

1. OrderService saves a PENDING order and publishes OrderCreated.
2. InventoryService listens to OrderCreated, reserves stock, and publishes InventoryReserved.
3. PaymentService listens to InventoryReserved, charges the card, and publishes PaymentSucceeded.
4. OrderService listens to PaymentSucceeded and changes the order to APPROVED.

If it fails: If PaymentService fails, it publishes PaymentFailed. The InventoryService listens for PaymentFailed and executes its compensation logic: un-reserving the stock.

Pros: Extremely loosely coupled. No single point of failure. Fast. Cons: For complex Sagas involving 5+ services, it becomes impossible to understand the flow just by looking at the code. You have created an emergent, implicit state machine that only exists in the network. If a bug occurs, tracking down why an order is stuck in a PENDING state is a distributed tracing nightmare.

2. Orchestration (Command-Driven)

In an Orchestration-based Saga, a central coordinator (the Orchestrator) explicitly tells the participating services what to do via asynchronous commands.

The Flow:

1. The OrderService creates an OrderSaga object in its database.
2. The OrderSaga sends a ReserveInventoryCommand to the InventoryService.
3. The InventoryService replies with an InventoryReservedReply.
4. The OrderSaga receives the reply, updates its internal state machine, and sends a ChargeCardCommand to the PaymentService.

If it fails: If the PaymentService replies with PaymentFailedReply, the OrderSaga immediately sends a CancelInventoryCommand to the InventoryService.

Pros: The entire business flow is defined in one place (the Saga code). It is extremely easy to monitor the state of the transaction. You avoid cyclical dependencies between services. Cons: The Orchestrator can become a god-class that contains too much domain logic.

The Semantic Lock Problem (Isolation)

Sagas lack the "I" in ACID: Isolation.

Because a Saga is a series of independent transactions, the intermediate state of the Saga is visible to other concurrent requests.

Imagine a Saga that reserves a seat on a flight and then charges the card. Between those two steps, the seat is marked as "reserved" in the database. Another user searching for flights will see that seat as unavailable. If the first user's credit card is declined, the compensation runs and frees the seat. But the second user already saw it as unavailable and booked a different flight.

This lack of isolation can lead to dirty reads and lost updates.

Mitigation: You must design your system to handle intermediate states. Instead of immediately deleting an order, you mark it as CANCELLING. Instead of immediately updating an inventory count, you use a pending_reservations table. This is often referred to as a Semantic Lock.

Conclusion

The Saga Pattern is the most complex architectural pattern in the microservices playbook. You should only use it when absolutely necessary.

Before implementing a Saga, ask yourself: Can I combine these two microservices back into a single service? If the answer is yes, do it. The pain of a slightly larger service is vastly preferable to the pain of managing distributed compensations.

But if the services must remain separate, the Orchestrated Saga provides the only reliable way to maintain data consistency across distributed boundaries without sacrificing availability.