Latency, CAP Theorem, and Consistency Models

Latency, CAP Theorem, and Consistency

CAP theorem is one of the most cited — and most misquoted — ideas in distributed systems. Most engineers can recite "consistency, availability, partition tolerance, pick two," and almost none can explain what that actually constrains in a system they're operating. This guide gives you the accurate version of CAP, the consistency models it interacts with, the PACELC extension that fixes CAP's biggest blind spot (latency), and — most importantly — how these trade-offs actually show up when you're picking a database or designing a cache for a real service.

1. What CAP Actually Constrains

CAP theorem, formalized by Eric Brewer and later proven by Gilbert and Lynch, states that a distributed data store can provide at most two of the following three guarantees simultaneously, during a network partition:

• Consistency (C) — every read receives the most recent write, or an error. This is linearizability, a strong guarantee — not the "C" in ACID.
• Availability (A) — every request to a non-failing node receives a (non-error) response, without guarantee it's the most recent write.
• Partition Tolerance (P) — the system continues to operate despite arbitrary network partitioning between nodes.

CP vs AP in practice

2. Consistency Models

"Consistency" is not binary. Distributed systems offer a spectrum of consistency models, each with different guarantees and different performance costs.

More guarantees, generally, cost more latency and availability. Fewer guarantees give you speed and resilience but push complexity onto the application.

Consistency models compared

Real bug pattern: cache/DB consistency

3. PACELC: The Extension CAP Doesn't Cover

CAP only describes trade-offs during a network partition. But partitions are rare; most of the time your system is running normally, and even then there's a trade-off between latency and consistency. PACELC, introduced by Daniel Abadi, makes this explicit:

If Partitioned: trade-off between Availability and Consistency; Else (normal operation): trade-off between Latency and Consistency.

This is a more honest model of real systems, because it captures the trade-off you're making every single request, not just during rare partition events. A system can be labeled with all four letters at once, e.g.:

4. How This Shows Up in Real Backend Decisions

Choosing a database

Latency budget example: synchronous replication cost

Single-region write, no replication ack required:      ~2ms  (local disk fsync)
Synchronous replication to 1 same-region replica:      ~2-5ms  (extra network round trip)
Synchronous quorum write across 3 AZs (same region):    ~5-15ms (round trip to majority)
Synchronous write requiring cross-region quorum:        ~50-150ms (WAN round trip)

This is the concrete cost of buying stronger consistency: every additional round trip you require before acknowledging a write is latency the client waits on. A cross-region synchronously-consistent write is often 10-50x slower than a local, asynchronously-replicated one — which is exactly why globally distributed systems lean AP/eventual for most data and reserve synchronous cross-region consensus for the handful of operations (leader election, critical financial state) that truly require it.

5. Common CAP Misconceptions

Because CAP gets summarized in a soundbite so often, it accumulates myths. Being able to correct these precisely is one of the fastest ways to signal real distributed-systems understanding in an interview.

6. Production Observations

• "Eventual consistency" without a consistency window is an incomplete spec. "Eventually" could mean 10 milliseconds or 10 minutes depending on replication lag under load. When you design with an AP store, state the expected replication lag as an SLO and monitor it — unbounded staleness is a silent correctness bug waiting to happen during a traffic spike or partial outage.
• Read replicas are a latency/consistency trade-off you make constantly, even in a boring single-region RDBMS setup. Reading from a replica to reduce primary load means accepting replication lag; a SELECT immediately after a related INSERT can miss the row if routed to a lagging replica.
• Idempotency keys are how you make AP systems safe for writes that must not duplicate (e.g., payment charges). Since AP systems favor availability under partition, a client may retry a write it's unsure succeeded — an idempotency key lets the server safely dedupe that retry instead of double-processing it.
• Distributed locks need a consensus-backed store, not a cache. Using Redis alone for a distributed lock without care for its failure modes (e.g., a primary failover losing an unreplicated lock) can produce two clients both believing they hold the lock. Redlock and similar algorithms exist to address this, and for genuinely critical locks (leader election), a CP system like ZooKeeper or etcd is the more defensible choice in a design interview.
• Vector clocks and version vectors are how AP systems detect conflicting concurrent writes without a central arbiter — worth naming if asked how Dynamo-style systems resolve divergent replicas (last-write-wins is the simpler but lossier alternative).
• Clock skew is a hidden tax on any timestamp-based consistency scheme. "Last write wins" using wall-clock timestamps assumes synchronized clocks across nodes; NTP drift of even tens of milliseconds can cause an actually-later write to lose to an actually-earlier one. Logical clocks (Lamport timestamps, vector clocks) sidestep this by tracking causality instead of wall-clock time.

Key takeaways

• CAP is not "pick two of three" — partition tolerance isn't optional in a real distributed system, so the actual choice is C vs A during a partition.
• Consistency is a spectrum (strong, sequential, causal, read-your-writes, monotonic, eventual), not a binary; pick the weakest model that's still correct for the data in question.
• PACELC captures the trade-off CAP misses: even with no partition, you trade latency for consistency on every request.
• Apply consistency choices per data type within a system, not as one global architectural stance — payment state and view counters have very different needs.
• Every synchronous replication hop you add to satisfy stronger consistency is a direct, measurable latency cost — quantify it before committing to it.
• Idempotency keys, version tokens, and short-TTL sticky reads are cheap, practical tools for buying back the UX cost of eventual consistency without paying for full strong consistency.
• "Eventually consistent" needs a bounded, monitored replication-lag SLO — otherwise it's an unbounded promise, not a design decision.

Interview Questions

• Explain CAP theorem accurately. Why is "pick two of three" a misleading way to state it?
• What is the difference between a CP and an AP system? Give a real database example of each and describe its behavior during a partition.
• What is PACELC, and why does it describe real-world trade-offs better than CAP alone?
• Define strong, causal, and eventual consistency. Order them by both guarantee strength and latency cost.
• What is "read-your-writes" consistency, and why does its absence cause a specific, user-visible bug? How would you fix it without going fully strongly consistent?
• Walk through a cache-aside pattern (update DB, then invalidate cache) and explain the race condition that can leave the cache stale.
• You're designing a shopping cart service and an account balance service in the same system. Would you use the same consistency model for both? Why or why not?
• How does synchronous cross-region replication affect write latency? Give rough relative numbers for local vs. same-region-multi-AZ vs. cross-region writes.
• Why do distributed locks typically require a CP system (like etcd or ZooKeeper) rather than a plain cache?
• What problem do idempotency keys solve in an eventually-consistent or at-least-once-delivery system?
• What is clock skew, and why can it break naive "last write wins" conflict resolution? What's an alternative?
• If a read replica is lagging behind the primary, what specific bugs can that cause for a user who just wrote data?
• How would you classify DynamoDB and a traditional single-primary RDBMS with synchronous replication under the PACELC model?
• What monitoring would you put in place to catch an AP system's "eventual" consistency window silently growing during a traffic spike?