Stage 3 · Build
Load Balancing & Traffic Management
Load Balancing Algorithms
Round robin, least connections, IP hash, consistent hashing, and EWMA.
Round Robin
Round robin sends each new connection to the next server in the list. Server 1 gets connection 1, server 2 gets connection 2, server 3 gets connection 3, server 1 gets connection 4, and so on. It is the simplest and most common algorithm.
Connections: C1 C2 C3 C4 C5 C6 C7 C8
Servers: S1 S2 S3 S1 S2 S3 S1 S2
Each server gets roughly equal connections.
Works best when all servers are identical and requests have similar cost.Round robin works well when servers are homogeneous and request costs are similar. It breaks down when one request is 100x more expensive than another — that request overloads whichever server gets it.
Least Connections
Least connections routes to the server with the fewest active connections. This naturally balances load when request durations vary — a server handling slow requests accumulates fewer new connections because it stays busy.
Server A: 3 active connections (handling slow queries)
Server B: 1 active connection (finished quickly)
Server C: 2 active connections
New request -> Server B (fewest connections)
Even though Server B already handled requests,
it finished faster and has more capacity.Least connections is better than round robin when request durations vary widely. However, it requires tracking active connections, which adds overhead. Most load balancers support both algorithms and let you choose per service.
IP Hash and Consistent Hashing
IP hash uses a hash of the client's IP to determine which server handles the request. The same client always goes to the same server (sticky sessions). This is useful for session state that cannot be shared across servers.
Client IPs: 10.0.0.1, 10.0.0.2, 10.0.0.3, 10.0.0.4
Servers: S1, S2, S3
hash(10.0.0.1) % 3 = 0 -> S1
hash(10.0.0.2) % 3 = 2 -> S3
hash(10.0.0.3) % 3 = 1 -> S2
hash(10.0.0.4) % 3 = 0 -> S1
Same client always hits same server.
Problem: adding/removing a server shuffles ALL mappings.Basic IP hash has a problem: when servers are added or removed, most clients get reassigned. Consistent hashing fixes this by using a hash ring — only a small fraction of clients are redistributed when the server pool changes.
Consistent hashing maps servers and clients onto a ring. A client's request goes to the next server clockwise on the ring. Adding or removing a server only affects clients between it and its neighbors on the ring. This is the foundation of distributed caches and CDNs.
Weighted Algorithms
Server A: weight 3 (powerful machine)
Server B: weight 1 (smaller machine)
Server C: weight 2 (medium machine)
Weighted round robin: A A A B C A A A B C ...
Server A gets 50% of traffic
Server C gets 33% of traffic
Server B gets 17% of trafficWeighted algorithms let you distribute traffic proportionally to server capacity. A server with weight 3 gets 3x more traffic than one with weight 1. This is essential when servers have different hardware or are serving different workloads.
EWMA and Latency-Based
Exponentially Weighted Moving Average (EWMA) tracks the average latency of each server. Requests are routed to the server with the lowest recent latency. This adapts in real-time to server performance changes.
Server A latency history: 10ms, 12ms, 8ms, 11ms
EWMA(A) = 0.7 * 11ms + 0.3 * 10ms = 10.7ms
Server B latency history: 50ms, 45ms, 60ms, 55ms
EWMA(B) = 0.7 * 55ms + 0.3 * 50ms = 53.5ms
New request -> Server A (lower EWMA latency)
If Server A starts slowing down:
Server A latency: 50ms, 60ms
EWMA(A) = 0.7 * 60ms + 0.3 * 50ms = 57ms
Traffic shifts to Server BEWMA is a weighted average that gives more importance to recent measurements. A weight of 0.7 means 70% of the average comes from the most recent measurement. This makes the algorithm responsive to changes while smoothing out spikes.
Round robin for homogeneous servers with similar request costs. Least connections for heterogeneous request durations. IP hash for sticky sessions. EWMA for performance-sensitive routing. Weighted when servers have different capacities.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.