The CAP Theorem is a key idea in distributed systems that explains why building a perfectly reliable, networked data system is impossible. When a network partition occurs, a distributed system can provide either Consistency or Availability, but not both simultaneously.
CAP Theorem Properties
1. Consistency
Consistency means that all the nodes (databases) inside a network will have the same copies of a replicated data item visible for various transactions. It guarantees that every node in a distributed cluster returns the same, most recent, and successful write. It refers to every client having the same view of the data. There are various types of consistency models.
Example: A user checks his account balance and knows that he has 500 rupees. He spends 200 rupees on some products. Hence the amount of 200 must be deducted, changing his account balance to 300 rupees.
- Both DB1 and DB2 show the same balance of 500.
- After spending 200, DB1 shows 300 and DB2 still shows 500.
- All replicas should reflect the same updated data.
2. Availability
Every request received by a non-failing node must receive a response, even if the response contains outdated data. The key word here is "every". In simple terms, every node (on either side of a network partition) must be able to respond in a reasonable amount of time.
Example: User A is a content creator having 1000 other users subscribed to his channel. Another user B who is far away from user A tries to subscribe to user A's channel.
- User B can subscribe to User A's channel even if updates are not immediately synchronized.
- Available: The subscription succeeds, updating the count from 1000 to 1001.
- Not Available: The subscription request fails, so the count remains 1000.
- If the system rejects the write request to preserve consistency, it sacrifices availability.
3. Partition Tolerance
Partition tolerance means that the system can continue operating even if the network connecting the nodes has a fault that results in two or more partitions, where the nodes in each partition can only communicate among each other. The system continues operating despite network failures. Whether it chooses consistency or availability depends on its design (CP or AP). Network partitions are a fact of life. Distributed systems guaranteeing partition tolerance can gracefully recover from partitions once the partition heals.Â
Example: Take the example of the same social media network where two users are trying to find the subscriber count of a particular channel. Due to some technical fault, a network outage occurs.
- A network partition breaks the communication between the primary database (DB1) and its replica (DB2).
- User 2 accesses the last replicated data (1000 subscribers) stored in DB2.
- Both databases continue operating independently, ensuring Partition Tolerance.
The CAP theorem states that distributed databases can have at most two of the three properties: consistency, availability, and partition tolerance. As a result, database systems prioritize only two properties at a time.
CAP Consistency vs ACID Consistency
- In CAP, consistency means data values are identical across all replicas in a distributed system.
- In ACID, consistency means a transaction preserves database integrity constraints defined by the schema.
The Trade-Offs in the CAP Theorem
The CAP theorem implies that a distributed system can only provide two out of three properties:
1. CA (Consistency and Availability)
These types of systems always accept the request to view or modify the data sent by the user and they are always responded with data which is consistent among all the database nodes of a big, distributed network.
However, such distributed systems are not realizable in real world because when network failure occurs, there are two options: Either send outdated data which was replicated moments ago before network failure or do not allow user to access the already slightly outdated data. If we choose first option, our system will become Available and if we choose second option our system will become Consistent.
- CA systems are practical only in non-distributed (single-node) environments because distributed systems must tolerate network partitions.
- Therefore, in distributed systems, either consistency or availability must be sacrificed during a partition.
Example databases: Standalone (single-node) MySQL and PostgreSQL systems provide consistency and availability because partition tolerance is not a concern.
2. AP (Availability and Partition Tolerance)
These types of systems are distributed in nature, ensuring that user requests to read or modify data continue to be processed even during a network partition.
The system prioritizes availability over consistency and may return stale (outdated) data until the partition is resolved. Such design choices are commonly used when building social media platforms, blogs, news websites, and other applications where continuous availability is more important than immediate consistency.
- Temporary data inconsistency is acceptable.
- Designed to remain accessible even during network partitions.
- Distributed across multiple nodes for reliability and fault tolerance.
Example databases: Amazon DynamoDB, Apache Cassandra, CouchDB.
3. CP (Consistency and Partition Tolerance)
These types of systems are distributed in nature, ensuring that all database nodes always return consistent and up-to-date data, even in the presence of a network partition, though some user requests may be delayed or rejected to maintain consistency.
The system prioritizes consistency over availability and does not allow users to read crucial data from the stored replica which was backed up prior to the occurrence of network partition. Consistency is chosen over availability for applications where latest data plays an important role. In a train ticket booking application, there is one seat which can be booked. A replica of the database is created, and it is sent to other nodes of the distributed system. A network outage occurs which causes the user connected to the partitioned node to fetch details from this replica.
- Used in scenarios where data consistency is critical (e.g., ticket booking, stock market application).
- Prevents stale or inconsistent data (e.g., double-booking the last seat).
- Ensures all users see accurate and up-to-date information.
Example databases: Apache HBase, MongoDB, Google Cloud Spanner.
