Microservices have emerged as a default choice for building scalable, resilient, and maintainable applications. Microservices are best suited for large, complex applications. Managing distributed transactions in a microservices architecture is a challenging yet crucial task. In this article, we would like to delve into managing the distributed transactions using saga pattern and how you can implement it successfully in your application.
Before going over the Saga pattern, let us understand what a transaction is and its key characteristics or principles.
What is a transaction?
A transaction refers to a series of actions/operations that are executed as a single unit of work. Transactions ensure that if one part of the operation fails, all changes made during the transaction can be rolled back to maintain data integrity.
Below are the key characteristics of a transaction.
- Atomicity: Transactions should be atomic, meaning that all operations within a transaction are executed entirely or not at all. If any part of the transaction fails, all changes should be rolled back to ensure data consistency.
- Consistency: The system must always be in a valid state before and after executing the transactions. i.e., the transaction brings the data only from one valid state to another valid state.
- Isolation: Transactions should be isolated from other concurrent transactions to prevent interference and maintain data integrity. Isolation guarantees that concurrent transactions produce the same data state that sequentially executed transactions would have produced.
- Durability: Once a transaction is committed, its effects should be permanent and survive system failures. This ensures that data changes are reliably stored and can be recovered if necessary.
What is a distributed transaction?
In a microservices ecosystem, transactions often span multiple services and systems. In a distributed transaction, multiple operations across different services or databases are treated as a single transaction to maintain data consistency and integrity across the entire system.
Saga Pattern
The Saga pattern provides transaction management by breaking down the distributed transactions into a sequence of local transactions. A local transaction represents the individual unit of work carried out by a stakeholder. Each local transaction modifies the database and emits a message or event to initiate the subsequent local transaction within the saga. The saga performs a sequence of compensating transactions to reverse the changes made by the previous local transactions if a local transaction fails.
Transactions that can be used to undo or offset the changes made by previously carried out transactions are known as compensating transactions.
The image below depicts the requests flow between microservices to fulfill a create order saga.
The Saga pattern can be implemented in two common ways. such as:
- Orchestration
- Choreography
Orchestration Pattern
The orchestration is a centralized approach to manage long-running distributed transactions across multiple services, or microservices. In the orchestration pattern, a central coordinator (or orchestrator) manages and coordinates the transaction flow by instructing each service involved in the transaction to execute specific steps, ensuring the overall transaction’s success or failure. i.e., the orchestrator executes saga requests, stores and interprets the states of each task, and handles failure recovery with compensating transactions.
The image below depicts the requests flow between microservices to fulfill a create order saga using an Orchestration pattern.
How it works?
- Central Coordinator (Orchestrator): A central coordinator (orchestrator) initiates and manages the distributed transaction by coordinating the execution of local transactions across multiple services or microservices involved in the transaction.
- Transaction Steps: The orchestrator defines and sequences the steps of the distributed transaction, instructing each service to perform its local transaction based on the predefined workflow.
- Local Transaction: Each service involved in the transaction executes its local transaction as instructed by the orchestrator, updating its local database and reporting the transaction’s status (completed, failed, compensated) back to the orchestrator.
- Compensation: If a failure occurs during the transaction, the orchestrator triggers compensating transactions to undo or compensate for the changes made by previously executed transactions, restoring the system to a consistent state.
Pros
- Well suited for complex workflows involving many participants or new participants added over time.
- Suitable when there is control over every participant in the process and control over the flow of activities.
- Doesn’t introduce cyclical dependencies, because the orchestrator unilaterally depends on the saga participants.
- Saga participants don’t need to know about commands from other participants. Clear separation of concerns simplifies business logic.
- Ensures data consistency by controlling the sequence of local transactions and compensating transactions, ensuring that the system remains in a consistent state throughout the transaction’s execution.
Cons
- Additional design complexity requires the implementation of a coordination logic.
- The central orchestrator can become a single point of failure, impacting the overall system’s availability and reliability. Implementing fault-tolerant and resilient orchestrators is essential to mitigate this risk.
- Centralizing transaction coordination and management can introduce tight coupling between the orchestrator and participating services, making the system less flexible and harder to evolve and scale.
- The orchestration pattern can introduce performance overhead due to synchronous communication and centralized coordination, potentially impacting system latency and throughput, especially in high-concurrency scenarios.
Rollback in orchestration
The coordinator should know how to rollback in case of failure. For this reason, the coordinator has to store a log of events for each flow and perform compensation transactions in each corresponding microservices when doing a rollback.
In summary, the orchestration pattern in Saga provides a centralized and controlled approach to managing distributed transactions across multiple services, offering simplicity, control, and consistency at the expense of potential single points of failure, increased complexity, and performance overhead. Careful design, implementation, and monitoring are essential to effectively leverage the orchestration pattern in distributed systems.
Choreography
The choreography is a decentralized approach to managing long-running distributed transactions across multiple services or microservices. Unlike the orchestration pattern, where a central coordinator manages the transaction flow, in the choreography pattern, each service involved in the transaction collaboratively manages its part of the transaction using events without a central coordinator.
The image below depicts the requests flow between microservices to fulfill a create order saga using a Choreography pattern.
How it works?
- Local Transactions: Each service involved in the transaction performs its local transaction, which updates its local database and publishes events to notify other services about the changes made.
- Event Propagation: After completing its local transaction, each service publishes events to inform other services about the changes and trigger subsequent transactions or compensating transactions if necessary.
- Event Handling: Services subscribe to relevant events published by other services to update their state accordingly and continue the transaction flow by performing their local transactions or compensating transactions based on the received events.
- Compensating Transactions: If a failure occurs during the transaction, compensating transactions are triggered to undo or compensate for the changes made by previously executed transactions to restore the system to a consistent state.
Pros
- Decoupling: enables loose coupling between services by decentralizing the transaction coordination, allowing services to evolve independently without affecting other services.
- Scalability: The decentralized nature of the choreography pattern allows for better scalability as services can be scaled independently to handle increased load and transaction volume.
- Best suitable for simple workflows that require few participants and don’t need coordination logic.
- Doesn’t require additional service implementation and maintenance.
- Doesn’t introduce a single point of failure, since the responsibilities are distributed across the saga participants.
Cons
- Complexity: implementing and managing choreographed sagas can be complex due to the asynchronous and decentralized nature of the transaction coordination, requiring careful design, monitoring, and error handling.
- Consistency: ensuring data consistency across distributed services and handling eventual consistency can be challenging, requiring careful orchestration of local transactions, event propagation, and compensating transactions to maintain a consistent system state.
- Monitoring and Observability: Comprehensive monitoring and observability tools are essential to track transaction flows, detect failures, troubleshoot issues, and ensure the overall health and performance of the system.
Rollback in Choreography Pattern
As there is no centralized coordinator in the Choreography pattern, corresponding microservices listen for failure events to be able to rollback.
In summary, the choreography pattern in Saga provides a decentralized and event-driven approach to managing distributed transactions across multiple services, enabling loose coupling, scalability, and resilience while requiring careful design and implementation to ensure data consistency and system reliability.
When to use this pattern?
Use the Saga pattern when you need to:
- Ensure data consistency in a distributed system without tight coupling.
- Roll back or compensate if one of the operations in the sequence fails.
The Saga pattern is less suitable for:
- Tightly coupled transactions
That’s all for today!
Thank you for taking the time to read this article. I hope you have enjoyed it. If you enjoyed it and would like to stay updated on various technology topics, please consider subscribing for more insightful content.
References:
https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/saga/saga
