p99 Latency Spiked at 2am
hardIncident scenarioCommonly asked~20 minHLD roundBar raiserHiring managerSenior / SDE-3AmazonUberRazorpayAtlassianFlipkartObservabilityResilienceCaching
The question, as asked
“You're on call. p99 latency on the checkout API just went from 200ms to 4 seconds. Error rate is flat. Walk me through what you do.”
Scope it before you design it
The prompt is vague on purpose. These are the clarifying questions a strong candidate asks first — answer them out loud before you touch the architecture.
- Ask what else moved. Error rate flat but latency up is a very specific signature — it rules out a lot immediately, so confirm it before theorising.
- Ask whether p50 moved or only p99. If p50 is flat, a subset of requests is affected, and finding what they share is the whole investigation.
- Ask what changed — deploys, config flags, migrations, dependency releases. At 2am the answer is often a scheduled job rather than a human.
- Ask whether traffic changed. Latency up with traffic flat is very different from latency up because volume tripled.
- Ask whether it's all endpoints or one. A single endpoint points at a query or a dependency; everything points at a shared resource.
- Ask whether it's all hosts or some. One bad host is a very different fix from a systemic problem, and it's quick to check.
- Establish the business impact early — is checkout failing, or just slow? That decides how aggressively you mitigate before you understand.
Your answer, first
Commit something before anything unlocks. Saved on this device as you go.
0 charactersExpand for room to draw
Framework hint
Sketch or write your approach above first. The hint gives you the structure, not the answer.