Stage 4 · Provision
Scalability & Capacity
Capacity Modeling
QPS, concurrency, Little's Law, saturation curves, and headroom targets for production services.
Why Capacity Model?
Capacity modeling answers: how many instances do I need to handle expected traffic? Without a model, you either over-provision (wasting money) or under-provision (risking outages). A capacity model provides a data-driven answer backed by measurements.
QPS Modeling
Step 1: Estimate peak QPS
Daily active users: 1M
Requests per user per day: 10
Peak-to-average ratio: 3x
Average QPS = 1M × 10 / 86400 = 116 QPS
Peak QPS = 116 × 3 = 348 QPS
Step 2: Measure single-instance capacity
Load test: 1 instance handles 500 QPS at p99 < 200ms
Step 3: Calculate instances needed
Instances = Peak QPS / Single-instance capacity
Instances = 348 / 500 = 0.7
Step 4: Apply headroom (30%)
Instances = 0.7 × 1.3 = 0.91 → Round up to 1
Result: 1 instance handles peak traffic with 30% headroomThe capacity model accounts for peak traffic, not average. The 30% headroom handles unexpected spikes and ensures the system is not running at 100% utilization.
Little's Law
Little's Law states: L = λ × W, where L is the average number of requests in the system, λ is the arrival rate (QPS), and W is the average time a request spends in the system (latency). This connects QPS, latency, and concurrency.
# Little's Law: L = λ × W
# L = concurrent requests in system
# λ = arrival rate (QPS)
# W = average response time (seconds)
# Example:
lambda_val = 100 # 100 QPS
W = 0.2 # 200ms average latency
L = lambda_val * W
# L = 100 × 0.2 = 20 concurrent requests
# If each request uses 1 thread:
# Need at least 20 threads to handle 100 QPS at 200ms latency
# At 80% utilization:
# Need 20 / 0.8 = 25 threadsLittle's Law helps you determine the number of concurrent connections, threads, or database connections needed. It is the bridge between QPS and resource requirements.
Saturation Curves
Saturation curves show how latency increases as utilization increases. At low utilization, latency is flat. As utilization approaches 100%, latency spikes exponentially. This is why you should target 70-80% utilization, not 100%.
Latency
│
│ ╱
│ ╱
│ ╱
│ ╱
│ ╱
│ ╱────
│ ╱────
│────
└──────────────────────── Utilization
0% 50% 70% 80% 90% 100%
At 70% utilization: latency is ~1.3x baseline
At 80% utilization: latency is ~2x baseline
At 90% utilization: latency is ~5x baseline
At 99% utilization: latency is ~100x baselineQueueing theory predicts that latency increases exponentially as utilization approaches 100%. This is why running at 100% CPU causes massive latency spikes.
Headroom Targets
| Resource | Target Utilization | Headroom |
|---|---|---|
| CPU | 70% | 30% |
| Memory | 80% | 20% |
| Disk IOPS | 70% | 30% |
| Network | 70% | 30% |
| Connections | 80% | 20% |
Building a Capacity Plan
- Estimate traffic — Current + projected growth for 12 months.
- Load test — Measure single-instance capacity at target latency.
- Apply Little's Law — Convert QPS to concurrency requirements.
- Calculate instances — Peak QPS / capacity per instance.
- Add headroom — 30% buffer for spikes and growth.
- Review quarterly — Re-measure and update the model.
Never estimate capacity without load testing. Load test your actual service with realistic traffic patterns. The load test results are the foundation of your capacity model.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.