Stage 3 · Build
Load Balancing & Traffic Management
Load Balancing Basics
L4 (TCP/IP) vs L7 (HTTP), virtual IPs, and ECMP routing.
Why Load Balance
Load balancing distributes incoming traffic across multiple servers. This provides three benefits: scalability (add more servers to handle more traffic), availability (if one server fails, traffic shifts to healthy servers), and efficiency (no single server is overwhelmed).
[Load Balancer]
/ | \
[Server 1] [Server 2] [Server 3]
10.0.1.1 10.0.1.2 10.0.1.3
Client -> Load Balancer (VIP: 10.0.0.100)
LB picks a healthy server and forwards traffic
Response goes back through LB (or directly via DSR)The load balancer presents a single Virtual IP (VIP) to the outside world. Clients connect to the VIP, and the load balancer distributes connections across backend servers. The client never knows which server handled its request.
L4 vs L7 Load Balancing
| Feature | L4 (Transport) | L7 (Application) |
|---|---|---|
| Operates on | TCP/UDP packets | HTTP requests |
| Visibility | IP + Port | Headers, URL, cookies, body |
| Performance | Fast (packet forwarding) | Slower (full TCP termination) |
| SSL termination | Passthrough or termination | Always terminates |
| Routing | IP/port based | URL, header, cookie based |
| Use case | Database, Redis, raw TCP | HTTP APIs, web apps |
L4 load balancers forward TCP/UDP packets without inspecting the payload. They are fast because they do not need to parse HTTP. L7 load balancers understand HTTP and can route based on URL path, headers, cookies, or request content.
Virtual IPs and Floating IPs
# The load balancer advertises a Virtual IP
# Clients send traffic to the VIP
# On Linux, you can simulate a VIP
sudo ip addr add 10.0.0.100/24 dev eth0
# The VIP is associated with a MAC address
# ARP replies come from the active load balancer
# When LB fails, the backup takes over the VIP (VRRP)VIPs are the foundation of load balancing. The client sees one IP (the VIP) but traffic is distributed across multiple real IPs. High-availability setups use VRRP (Virtual Router Redundancy Protocol) to fail over the VIP if the primary LB dies.
ECMP Routing
Equal-Cost Multi-Path (ECMP) routing distributes traffic across multiple paths with equal cost. Instead of a single load balancer, ECMP uses multiple routers. BGP announces the same VIP from multiple routers, and the network hashes connections across them.
[Router A] ---- [Router B]
| |
[Server 1] [Server 2]
VIP announced from both Router A and Router B
Network hashes packets across both paths
Each server receives roughly half the traffic
No single load balancer bottleneck
Horizontal scaling of the load balancing layer itselfECMP scales the load balancing layer itself. Cloud providers use this approach: AWS NLB uses ECMP across multiple nodes. The trade-off is that ECMP uses flow-based hashing — a single long-lived TCP connection always goes to the same router.
Load Balancer Topology
Single tier (small scale):
[Client] -> [HAProxy] -> [Servers]
Two tier (typical production):
[Client] -> [Cloud LB (L4)] -> [Nginx (L7)] -> [Servers]
Three tier (high traffic):
[Client] -> [Cloud LB] -> [L4 LB pool] -> [L7 LB pool] -> [Servers]
Kubernetes:
[Client] -> [Cloud LB] -> [Ingress Controller] -> [Services] -> [Pods]Most production systems use multiple tiers. The outer tier handles TLS termination and basic routing. The inner tier handles content-based routing, rate limiting, and application-specific logic. Each tier adds latency but improves flexibility.
Always put an L4 load balancer in front of your L7 load balancers. L4 handles high-volume TCP forwarding efficiently. L7 does the smart routing but at higher CPU cost. This two-tier approach gives you both performance and flexibility.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.