System Design Problem Solving: A Practical Framework

System Design Problem Solving

Everything in the two previous guides — scaling strategy, load balancing, CAP trade-offs, consistency models — is raw material. This guide is about assembling that material into a coherent answer under interview conditions, in 35-45 minutes, without going in circles. The framework below is not the only valid approach, but it is a reliable default: it forces you to establish scope and scale before you draw a single box, and it leaves time for the deep dive that actually differentiates a strong answer from a mediocre one.

1. The Framework

Time allocation for a 40-45 minute interview

2. Step 1 — Clarify Requirements

Split requirements into functional (what the system does) and non-functional (how well it does it — the qualities that actually drive the architecture). Non-functional requirements are what separate system design from a CRUD app tutorial.

Questions to ask, every time

3. Step 2 — Estimate Scale

Convert the requirements into rough numbers for traffic, storage, and bandwidth. The goal isn't precision — it's getting within an order of magnitude so you can make defensible decisions (do we need sharding? do we need a CDN? does a single Postgres instance suffice?).

The estimation checklist

Traffic:
  - Requests/second (average and peak)
  - Read : write ratio

Storage:
  - Bytes per record × number of records = total storage
  - Growth rate per year

Bandwidth:
  - Requests/sec × average payload size = throughput

Memory (for caching):
  - Working set size (hot data) → cache sizing

4. Step 3 — High-Level Design

Draw the major components and the request flow through them, using the vocabulary from the scalability and CAP guides in this phase: where's the load balancer, is the service tier stateless, where does caching sit, what's the data store and why, is anything async (queue-backed).

5. Step 4 — Deep Dive

Pick the 1-2 components that are actually interesting for this problem — usually wherever the scale, consistency, or latency requirements create real tension — and go deep: schema design, algorithm choice, specific trade-offs, failure handling. This is where the interview is actually won or lost; the high-level design is table stakes.

6. Step 5 — Trade-offs and Bottlenecks

Close by naming what breaks first as scale grows, and what you'd change. This demonstrates you understand your own design's limits rather than presenting it as a finished, flawless artifact.

Worked Example 1: URL Shortener

A classic because it's small enough to fully design in 40 minutes but touches ID generation, caching, read-heavy scaling, and redirect latency — a good showcase of the full framework.

Clarify requirements

• Functional: shorten a long URL → short code; redirecting a short code → 302 to the original URL; optional custom aliases; optional expiration.
• Non-functional: extremely read-heavy (redirects vastly outnumber creations); redirect latency should be very low (sub-100ms, ideally sub-20ms with caching); short codes must be unique; eventual consistency for creation-to-availability is acceptable (a few seconds of propagation delay is fine).

Estimate scale

Assume: 100M new short URLs created per month, 100:1 read:write ratio

Writes/sec (avg) = 100,000,000 / (30 × 86,400) ≈ 39 writes/sec
Reads/sec (avg)  = 39 × 100 ≈ 3,900 reads/sec
Peak reads (3x)  ≈ 11,700 reads/sec

Storage per URL: short_code(7B) + long_url(~500B) + metadata(~50B) ≈ 560B
5-year storage: 100M/mo × 60 months × 560B ≈ 3.36 TB  (small — fits comfortably
                                                          on modern SSD-backed DBs,
                                                          sharding not yet required)

Short code space: base62 (a-z, A-Z, 0-9), 7 characters
  62^7 ≈ 3.5 trillion possible codes — far more than needed

High-level design

Deep dive: ID generation strategy

This is the component worth going deep on — the choice materially affects correctness and scalability.

Deep dive: caching the redirect path

Since reads dominate 100:1, the redirect path is where latency and infrastructure cost are actually won or lost.

• Cache-aside with Redis: on a cache miss, read from DB and populate the cache; TTL of a few hours to a day, since URLs rarely change once created.
• Cache the hot subset, not everything: real-world short URL access follows a power law — a small fraction of links get the vast majority of clicks. Size the Redis cluster for the hot working set, not the full 3.36 TB dataset; even a few GB of cache captures the overwhelming majority of redirect traffic.
• CDN edge caching for the redirect response itself is viable since a 302 for a given short code changes rarely — pushes cache hits even closer to the user and off your own infrastructure entirely for very hot links.

Trade-offs and bottlenecks at 10x scale

• At 10x write volume, a single auto-increment counter becomes the bottleneck — move to range-based ID allocation (each app server reserves a block of 10,000 IDs at a time) to cut DB round trips per write by four orders of magnitude.
• At 10x read volume, add more cache replicas and consider a consistent-hashing cache cluster rather than a single Redis instance, to avoid a single cache node becoming a hotspot.
• Analytics (click counts per link) is intentionally deferred — bolting it onto the synchronous redirect path would add latency to the hot path; the correct answer is to fire an async event (queue) on redirect and aggregate click counts out-of-band.

Worked Example 2: Order and Notification System

A less "toy" example than a URL shortener — this is closer to a real capstone-style backend system and forces you to reason about consistency and asynchronous workflows together, tying back to the messaging and persistence phases earlier in this roadmap.

Clarify requirements

• Functional: a customer places an order; the system reserves inventory, processes payment, persists the order, and notifies the customer (email/SMS/push) of order confirmation and later status changes (shipped, delivered).
• Non-functional: order placement and payment must be strongly consistent (no double-charging, no overselling inventory); notification delivery can be eventually consistent — a notification arriving a few seconds late is fine, and at-least-once delivery (occasional duplicate notification) is acceptable, but losing a notification silently is not.

Estimate scale

Assume: 500,000 orders/day, notifications average 3 per order (confirmation,
        shipped, delivered)

Orders/sec (avg) = 500,000 / 86,400 ≈ 5.8 orders/sec
Peak (5x, e.g. sale event) ≈ 29 orders/sec

Notifications/sec (avg) = 5.8 × 3 ≈ 17.4/sec
Notifications/sec (peak) ≈ 87/sec

High-level design

Deep dive: consistency boundary between order placement and notification

This is the crux of the design. Order placement (inventory + payment) and notification delivery have fundamentally different consistency requirements, and the architecture should reflect that split explicitly rather than treating the whole flow as one transaction.

Deep dive: handling partial failure

Deep dive: notification delivery guarantees

• At-least-once delivery, deduplicated on the client side where possible: the notification service should assume a message may be redelivered (consumer crash after processing but before acknowledging the queue message) and design notifications to be safe to send twice — or track a delivery-attempted flag keyed by (order_id, notification_type) to skip true duplicates.
• Retry with exponential backoff for transient provider failures (email/SMS gateway briefly unavailable), and a dead-letter queue after N failed attempts so a permanently-failing notification doesn't block the queue or get silently dropped — it gets surfaced for investigation instead.
• Fan-out per channel: treat email, SMS, and push as independent, parallel deliveries rather than a single sequential pipeline, so a slow or down SMS gateway doesn't delay the email that would otherwise have gone out immediately.

Trade-offs and bottlenecks at 10x scale

• At 10x order volume, the single strongly-consistent order/inventory database becomes the constrained resource — the mitigation is sharding by a key like customer_id or warehouse_id (whichever aligns with how inventory is actually partitioned in the business), not weakening the consistency guarantee, since overselling remains unacceptable at any scale.
• The notification service, being stateless and queue-driven, scales horizontally by simply adding more consumers — this is intentionally the easy part of the design, which is exactly why the strongly consistent order core deserved the deeper design attention.
• At extreme scale, consider read models: a denormalized, eventually-consistent "order history" read view (updated via the same outbox events) so customer-facing order-status queries don't compete for capacity with the transactional write path.

7. Common Mistakes That Sink an Otherwise Good Design

Key takeaways

• The framework — clarify, estimate, design, deep-dive, trade-offs — exists to force requirements and scale to drive the architecture, not the other way around.
• Non-functional requirements (latency, consistency, availability, durability) are what actually differentiate one valid design from another; functional requirements alone rarely do.
• Back-of-envelope estimates should change at least one concrete decision (do we need sharding, caching, a queue) — an estimate that doesn't is a wasted five minutes.
• The deep dive is where interviews are won — pick the 1-2 components with genuine design tension for this specific problem, not the ones you rehearsed.
• Different data within the same system can and should have different consistency guarantees — a strongly consistent order core and an eventually consistent notification pipeline are not a contradiction, they're good design.
• The transactional outbox pattern is the standard, defensible answer to "how do you atomically commit state and publish an event about it."
• Idempotency keys are the answer to "what if the network fails between the request and the response" for any operation that must not be duplicated.
• Always close by naming what breaks first at 10x scale — it demonstrates you understand your design's limits rather than presenting a false sense of completeness.

Interview Questions

• Walk through the framework you use to approach a system design interview question, end to end.
• Why is it important to clarify non-functional requirements before starting the high-level design?
• How would you estimate the number of requests per second a system needs to handle, given daily active users and an average actions-per-user number?
• Design a URL shortener. What ID generation strategy would you use, and how would your answer change at 1,000x the write volume?
• Why does a URL shortener's design lean so heavily on caching the redirect path specifically, rather than the creation path?
• Design an order placement system that also sends order-status notifications. Which parts need strong consistency and which can be eventually consistent, and why?
• What is the transactional outbox pattern, and what specific bug does it prevent?
• How would you handle a payment gateway request that times out — where you don't know if the charge succeeded?
• What does "at-least-once delivery" mean for a notification system, and how do you make duplicate notifications safe?
• What would you change about your order system's design if order volume grew 10x?
• How do you decide which 1-2 components deserve a deep dive in a 40-minute design interview?
• What's a concrete example of over-engineering a design, and how would you recognize you're doing it mid-interview?
• Why is "we'll use a message queue" not a complete answer on its own — what follow-up questions should you expect and be ready to answer?
• How would you explain, using the CAP/PACELC vocabulary, why an order's payment step is synchronous but its notification step is asynchronous?