CQRS (command query responsibility segregation)

What is CQRS (command query responsibility segregation)?

Command query responsibility segregation (CQRS) is a programming design and architectural pattern that treats retrieving data and changing data differently. CQRS uses command handlers to simplify the query process and hide complex, multisystem changes.

The design pattern was developed by Bertrand Meyer as a way to ensure a method working with data is only able to perform one of two tasks. A method can either retrieve information or it can modify it, but it can't do both. In object-oriented terms, this paradigm separates responsibilities into two different classes -- one for read and the other for delete, create and update.

When CQRS is combined with event sourcing, it guarantees an audit log of changes to the database that maintains transactional consistency.

When to use the CQRS pattern

The CQRS pattern is useful in a few specific scenarios:

• Collaborative domains. The primary application of CQRS is in collaborative domains, where numerous users can view the same data concurrently.
• Task-based user interfaces. It's also helpful with task-based user interfaces, where users are led through a series of steps to finish a challenging task.
• High-traffic systems. With the help of CQRS, better performance and scalability can be achieved by evenly distributing the workload between read and write operations.
• Querying from repositories. CQRS is useful when it's challenging to query all the data that users need to view from repositories.
• Complex business logic. CQRS can assist in streamlining the design of an application if it contains complicated business logic by separating the read and write processes.
• Optimizing read operations. For applications that have a large number of read operations, CQRS can optimize those operations by creating dedicated read models.
• Different data models for reads and writes. In some instances, the data model used for writing to the database might not be appropriate for querying. Through CQRS, multiple data models can be established for each operation by separating the read and write models.
• Support for event sourcing. CQRS can be combined with event sourcing patterns to develop a system that can handle a large number of events and queries.

It's crucial to remember that CQRS should only be used in restricted situations where it's truly necessary. It's useless for systems that follow the CRUD -- create, read, update and delete -- mental model.

The core concept of CQRS is to separate reads and writes into different models and to analyze the actions of a particular area. When deploying CQRS, it's critical to consider the unique requirements and limitations of a system, and thoroughly assess whether this pattern is appropriate for the particular use case.

How the CQRS pattern is executed

The most common way to deploy CQRS is the command pattern, which is the software system that defines a high-level interface. At runtime, the base class takes the command, creates the appropriate object handler -- perhaps update, delete or create -- and calls a method to execute the command.

Before and after the execution, the base class can log that the method was called. This creates a log that can be replayed from and to any point in time. Once the interface and dispatch code exist, the computer program to take the events and replay them is not much more complex than a for loop. That is, the program reads a file from a certain point, and then for each line, it calls the command on that line with the data on that line. There is a great deal of complexity hidden inside the command handler, but it is limited, or encapsulated by the handler.

The handlers themselves call each step of the process to create, update or delete a logical item in the system. They can dispatch the first request, but also monitor to see when that action completes, handling errors and rollbacks if needed. That means the handler sets up a long-running, two-phase commit, sometimes called a saga pattern. When the handler completes -- or the transaction fails and errors -- the handler will write the results to the event log.

A system that creates and tracks changes in this way will eventually create an event log with a record of every system change. If this log is created in a consistent way that a program can read and replay, it enables event sourcing.

CQRS example

An example of CQRS can be seen during a customer's order process. When the customer looks at an order, the process is relatively straightforward: a read from a database. In practice, that read could be from a NoSQL key-value store, such as Redis. This key-value stores all the easy-to-access information about the customer in one place. A microservice picks up the information, and a webpage displays it. The read side is simple and can be done entirely by the Redis team working with the front-end developers. Thus, the responsibility for the read of CQRS is separated from the write.

The write side of the operation, however, is much more complex and contains several different steps and dependencies. For example, if an order is canceled before it is shipped, the return needs to be canceled in the cache and needs to be returned in the main source system of record. Companies with a data warehouse will need to delete the record from the warehouse as well as other auxiliary systems. The physical warehouse and shipping need to be notified to not ship the item. Other complexities can include the credit card charges needing to be reversed, inventory counts in the warehouse needing to be changed and the refill order from the supplier needing to go down by one.

Because a single change needs to be copied to so many places, the update, delete and create side of the operation will need to interact with either an enterprise service bus or a command handler, such as the saga pattern. CQRS provides a place to put the logic to handle these complex transactions.

Challenges of CQRS

CQRS typically comes with the following challenges:

• Complexity. The main challenge of CQRS is also, in a way, its benefit as it creates a layer of complexity on top of the existing codebase. Instead of a simple relational database such as SQL, now the application will have command handlers, dispatching and logging. All this additional code creates the potential for error.
• Messaging. CQRS doesn't require messaging, but it's typical to use messaging to process commands and broadcast update events. In that case, the application must handle message failures or duplicate messages.
• Conversion. Because CQRS solves a complexity problem, it appeals to organizations that already have an existing system that is becoming more complex. That existing system might not have such a complex log of transitions or might use overnight batch mode to do updates. Conversion of such a currently online, running system to CQRS can be a challenge.
• Eventual consistency. If the read and write databases are separated, the read data can become stale. Therefore, the reading database must be updated to reflect any changes made in the writing database. However, it can cause data consistency issues as it can be difficult to detect when a user has issued a request based on outdated data from the reading database.
• Delayed data synchronization. Since the two models are independent of each other, changes made in the write mode can take some time to reflect in the read mode. This can lead to issues if a user queries the read mode before the changes have been synchronized.
• Cost. CQRS patterns sometimes come with a higher inherent cost in terms of hardware. Utilization costs can also go up if a cloud provider is used, since additional database technologies are required for deployment.

CQRS and event sourcing

The CQRS pattern is frequently used along with the event sourcing pattern. CQRS-based systems use separate read and write data models, where each model is tailored to relevant tasks and often located in physically separate data stores. Meanwhile, an event source pattern is an approach to a series or sequence of events.

The system keeps track of past changes in data states and alerts users on how to handle them. In event sourcing, every step of the data transformation process is recorded, not just the most recent state. Any user-generated action, such as clicking a mouse button or pressing a keyboard key, counts as an event.

Event sourcing suggests that the system itself can be re-created as the collection of all the changes over time. That creates a new view of data within a software system: the transaction log. With event sourcing, the log is more than a text file -- it's an actual set of commands that can be replayed to re-create the change.

In some cases, the event log can be "played backward" to undo large changes. Setting up event sourcing requires capturing the changes, along with building the software to replay those changes. Deployed carefully, the command handlers can write to a log, creating the transaction log for event sourcing. Running each line in the log through a command handler actually executes event sourcing. The two patterns have considerable overlap. The command in CQRS is essentially one way to create event sourcing.

What are some related patterns?

CQRS aligns the work of developers to what happens in the organization when, for example, an order is created. This is compatible with the approaches of domain-driven design -- an explicit domain model -- event sourcing and the command pattern. The messages to perform the create, update and delete might be delivered by an enterprise service bus, while the general pattern for architecture is service-oriented architecture.

While CQRS can be very useful in constructing an event-driven architecture, bad practices can turn it into more of a nuisance. Explore the three common CQRS pattern problems and see how they can be mitigated.