Course outline

01 · Designing for High Availability

02 · Load Balancing & Traffic Management

03 · Database Scaling Strategies

04 · Observability-Driven Design

05 · Data Systems at Scale

06 · System Design Interviews

07 · Asynchronous Systems

08 · Resilience Patterns

09 · Scalability & Capacity

10 · Cache Design Patterns

11 · Design Case Studies

Stage 4 · Provision

Scalability & Capacity

Autoscaling Signals

CPU, queue depth, custom metrics, KEDA, and Kubernetes HPA stabilization windows.

7 min readSystem Design for SREProvision

Why Autoscale?

Autoscaling adjusts the number of instances based on demand. It maintains performance during traffic spikes and reduces cost during quiet periods. Without autoscaling, you must provision for peak traffic 24/7, wasting resources during off-hours.

Kubernetes HPA

HPA configuration


apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 120

HPA scales up quickly (50% increase per minute) and scales down slowly (25% decrease per 2 minutes with 5-minute stabilization). This prevents oscillation.

Scaling Signals

Signal	Best For	Limitation
CPU	Compute-bound services	Not useful for I/O-bound services
Memory	Memory-heavy workloads	Does not scale down well
Queue depth	Async workers	Requires queue metrics
Request rate	HTTP services	Does not account for complexity
Custom metrics	Any workload	Requires metrics setup

Custom Metrics

HPA with custom metrics


apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 2
  maxReplicas: 50
  metrics:
    - type: Pods
      pods:
        metric:
          name: queue_depth
        target:
          type: AverageValue
          averageValue: "100"  # Scale when queue_depth > 100 per pod
    - type: Object
      object:
        describedObject:
          apiVersion: networking.k8s.io/v1
          kind: Ingress
          name: api-ingress
        metric:
          name: requests_per_second
        target:
          type: Value
          value: "1000"  # Scale when total RPS > 1000

Custom metrics allow scaling on any signal: queue depth, request rate, error rate, or business metrics. The metrics must be exposed via Prometheus adapter.

KEDA

KEDA (Kubernetes Event-Driven Autoscaling) extends HPA with event-source-aware scaling. It can scale based on Kafka consumer lag, SQS queue depth, Prometheus queries, and 50+ other scalers. KEDA scales to zero when idle, which HPA cannot do.

KEDA Kafka scaler


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-worker
spec:
  scaleTargetRef:
    name: order-worker
  pollingInterval: 15
  cooldownPeriod: 300
  minReplicaCount: 0    # Scale to zero when idle
  maxReplicaCount: 100
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: kafka:9092
        consumerGroup: order-processor
        topic: order-events
        lagThreshold: "50"    # Scale when lag > 50
        activationLagThreshold: "10"

KEDA scales the order-worker from 0 to 100 based on Kafka consumer lag. When lag is below 10, it scales to zero. This saves cost during idle periods.

Stabilization Windows

Stabilization windows prevent scaling thrash. Without them, HPA scales up and down rapidly in response to short-lived spikes. The scale-up window should be short (60s) for fast response. The scale-down window should be long (300s) to avoid premature scale-down.

Scale up fast, scale down slow

Set scale-up stabilization to 60s and scale-down to 300s. This ensures fast response to traffic spikes while avoiding cost-inefficient oscillation during quiet periods.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.