Autoscaling Strategy

HPA targets, KEDA triggers, predictive scaling, cooldowns, and scale-to-zero risks.

7 min readSRE PlaybookOperate

Autoscaling Overview

Autoscaling automatically adjusts capacity based on demand. It eliminates manual scaling, reduces toil, and improves reliability. But autoscaling must be configured correctly — poor configuration causes oscillations, lag, and outages.

Autoscaling is not set and forget

Autoscaling requires monitoring, tuning, and testing. A misconfigured autoscaler can cause more problems than manual scaling. Test your autoscaling configuration under load before relying on it in production.

HPA Targets

The Horizontal Pod Autoscaler scales pod count based on metrics. Choose targets that reflect actual bottlenecks. CPU is common but not always the right metric. Use custom metrics when CPU does not correlate with load.

HPA configuration with multiple metrics


apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: user-api-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: user-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    # CPU-based scaling
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

    # Memory-based scaling
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

    # Custom metric: requests per second
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

    # Custom metric: queue depth
    - type: External
      external:
        metric:
          name: queue_depth
          selector:
            matchLabels:
              queue: user-api
        target:
          type: AverageValue
          averageValue: "50"

  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
        - type: Pods
          value: 4
          periodSeconds: 60
      selectPolicy: Max

    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 120
      selectPolicy: Min

KEDA Triggers

KEDA extends autoscaling beyond Kubernetes metrics. It scales based on external systems: message queues, cron schedules, Prometheus metrics, and custom adapters. KEDA is essential for event-driven and batch workloads.

KEDA ScaledObject with multiple triggers


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: event-processor
  namespace: production
spec:
  scaleTargetRef:
    name: event-processor
  minReplicaCount: 2
  maxReplicaCount: 50
  cooldownPeriod: 300
  pollingInterval: 15
  triggers:
    # Scale based on RabbitMQ queue depth
    - type: rabbitmq
      metadata:
        queueName: events
        queueLength: "50"
        host: "amqp://rabbitmq.internal"

    # Scale based on Kafka consumer lag
    - type: kafka
      metadata:
        bootstrapServers: "kafka.internal:9092"
        consumerGroup: "event-processor"
        topic: events
        lagThreshold: "100"

    # Scale based on cron schedule
    - type: cron
      metadata:
        timezone: America/New_York
        start: "0 2 * * *"
        end: "0 6 * * *"
        desiredReplicas: "10"

    # Scale based on Prometheus metric
    - type: prometheus
      metadata:
        serverAddress: "http://prometheus.internal:9090"
        metricName: event_processing_rate
        threshold: "100"
        query: |
          sum(rate(events_processed_total[5m]))

Predictive Scaling

Predictive scaling uses historical patterns to anticipate demand. It scales up before traffic increases and scales down before traffic decreases. This eliminates the lag between traffic change and scaling response.

Predictive scaling with KEDA cron triggers


predictive_scaling:
  strategy: "cron_based"
  patterns:
    weekday_morning_ramp:
      schedule: "0 7 * * 1-5"
      scale_to: 8
      duration: "12h"
      description: "Weekday morning traffic ramp"

    evening_peak:
      schedule: "0 17 * * *"
      scale_to: 12
      duration: "5h"
      description: "Evening traffic peak"

    overnight_valley:
      schedule: "0 23 * * *"
      scale_to: 2
      duration: "8h"
      description: "Overnight low traffic"

  ml_based:
    tool: "Kubernetes Cluster Autoscaler with predictive scaling"
    training_data: "Last 90 days of traffic data"
    prediction_window: "24 hours"
    confidence_threshold: 0.85
    update_frequency: "1h"

Cooldowns

Cooldowns prevent autoscaling oscillations. Without cooldowns, the autoscaler scales up, then immediately scales down, then scales up again. This churn wastes resources and destabilizes the service.

Cooldown configuration


cooldown_settings:
  scale_up:
    stabilization_window: 60s
    description: "Wait 60 seconds before scaling up again"
    reason: "Prevent premature scale-up on transient spikes"

  scale_down:
    stabilization_window: 300s
    description: "Wait 5 minutes before scaling down"
    reason: "Traffic may spike again shortly after decrease"

  deployment_strategy:
    max_surge: "25%"
    max_unavailable: "0"
    description: "Rolling update ensures capacity during scale events"

  best_practices:
    - "Scale up fast, scale down slow"
    - "Use multiple metrics to avoid false triggers"
    - "Set minReplicas to handle baseline traffic"
    - "Test autoscaling with load tests before production"

Scale-to-Zero Risks

Scale-to-zero eliminates costs when there is no traffic. But it introduces cold start latency, which can cause user-facing issues. Use scale-to-zero only for non-user-facing services or services with infrequent traffic.

Scale-to-zero configuration with KEDA


apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: batch-processor
  namespace: production
spec:
  scaleTargetRef:
    name: batch-processor
  minReplicaCount: 0  # Scale to zero
  maxReplicaCount: 20
  cooldownPeriod: 600  # 10 minutes before scaling to zero
  triggers:
    - type: rabbitmq
      metadata:
        queueName: batch-jobs
        queueLength: "1"

risks:
  cold_start:
    description: "First request after scale-to-zero has latency spike"
    mitigation: "Use pre-warming, init containers, or keep minReplicaCount=1"
    impact: "5-30 second cold start delay"

  event_loss:
    description: "Events may be lost during scale-to-zero"
    mitigation: "Ensure message queue persists events"
    impact: "Events buffered in queue until pods start"

  cascade:
    description: "Multiple services scaling to zero simultaneously"
    mitigation: "Stagger cooldown periods, avoid chain dependencies"
    impact: "All services cold-start at once"

Test autoscaling under load

Autoscaling configuration must be tested under realistic load. A misconfigured HPA or KEDA trigger can cause oscillations, lag, or complete failure. Run load tests that exercise your autoscaling before relying on it in production.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.

Stage 6 · Operate

Capacity & Performance Engineering