Stage 6 · Operate

Capacity & Performance Engineering

Cost & Efficiency

Rightsizing, bin packing, spot capacity, reservation planning, and unit economics.

6 min readSRE PlaybookOperate

Cost Optimization

Cloud costs grow linearly with resource consumption. Cost optimization is not about cutting corners — it is about paying only for what you need. Over-provisioning wastes money that could fund reliability improvements.

Cost is a reliability input

Money saved on infrastructure can fund more engineers, better tooling, and game days. Cost efficiency is not at odds with reliability — it funds it.

Rightsizing

Rightsizing matches resource allocation to actual usage. Most services are over-provisioned. Analyze actual CPU and memory usage, then adjust requests and limits to match.

Rightsizing recommendations


rightsizing:
  method: "VPA recommendations + actual usage analysis"
  tools:
    - "Kubernetes VPA in recommendation mode"
    - "Grafana resource dashboards"
    - "Cloud provider cost explorer"

  analysis:
    cpu:
      method: "Compare requested vs actual usage"
      rule: "If actual p99 < 50% of requested, reduce request"
      example:
        requested: "500m"
        actual_p99: "200m"
        recommendation: "250m request, 500m limit"

    memory:
      method: "Compare requested vs actual peak usage"
      rule: "If actual peak < 70% of requested, reduce request"
      example:
        requested: "512Mi"
        actual_peak: "300Mi"
        recommendation: "384Mi request, 512Mi limit"

  savings_potential:
    typical_reduction: "30-50% of resource requests"
    monthly_savings: "Depends on current over-provisioning"
    risk: "Set limits higher than requests to handle spikes"

Bin Packing

Bin packing optimizes how pods are distributed across nodes. Better bin packing means fewer nodes, lower costs, and the same reliability. Kubernetes scheduler does this automatically, but you can influence it.

Bin packing optimization


bin_packing:
  strategy: "Schedule pods to fill nodes completely"
  tools:
    - "Kubernetes scheduler with MostAllocated scoring"
    - "Cluster autoscaler with expander=least-waste"

  configuration:
    # Scheduler profile for bin packing
    scheduler_config:
      profiles:
        - schedulerName: default-scheduler
          pluginConfig:
            - name: NodeResourcesFit
              args:
                scoringStrategy:
                  type: LeastAllocated
                  resources:
                    - name: cpu
                      weight: 1
                    - name: memory
                      weight: 1

  benefits:
    - "Fewer nodes required"
    - "Lower cluster infrastructure cost"
    - "Better resource utilization"
    - "Easier to maintain spare capacity"

Spot Capacity

Spot instances provide 60-90% cost savings for fault-tolerant workloads. Use spot for batch processing, stateless services, and non-critical workloads. Avoid spot for databases and stateful services.

Spot instance configuration


spot_capacity:
  suitable_workloads:
    - "Stateless API servers"
    - "Batch processing jobs"
    - "CI/CD runners"
    - "Caching layers"
    - "Development environments"

  unsuitable_workloads:
    - "Databases (unless replicated)"
    - "Stateful services"
    - "Critical singleton services"
    - "Services requiring guaranteed uptime"

  configuration:
    node_pool:
      name: "spot-pool"
      instance_types:
        - "m5.large"
        - "m5a.large"
        - "m4.large"
      capacity_type: "spot"
      taints:
        - key: "spot"
          value: "true"
          effect: "NoSchedule"

    pod_configuration:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: "spot"
                    operator: "In"
                    values: ["true"]
      tolerations:
        - key: "spot"
          operator: "Equal"
          value: "true"
          effect: "NoSchedule"

  cost_savings:
    typical_discount: "60-80%"
    interruption_rate: "Varies by region and instance type"
    mitigation: "Use multiple instance types, graceful shutdown"

Reservation Planning

Reserved instances and committed use discounts provide significant savings for predictable workloads. Analyze your usage patterns and commit to 1-3 year terms for baseline capacity.

Reservation strategy


reservation_strategy:
  baseline_capacity:
    description: "Capacity you use 24/7/365"
    reservation_type: "Reserved Instance or Committed Use"
    term: "1-3 years"
    savings: "30-60%"

  peak_capacity:
    description: "Capacity you use during peak hours"
    reservation_type: "Scheduled Reserved Instance"
    term: "1 year"
    savings: "20-40"

  burst_capacity:
    description: "Capacity for traffic spikes"
    reservation_type: "On-demand or Spot"
    savings: "0-80%"

  planning_process:
    - "Analyze 3 months of usage data"
    - "Identify baseline, peak, and burst patterns"
    - "Commit to baseline with reservations"
    - "Use on-demand for peak"
    - "Use spot for burst"
    - "Review quarterly and adjust"

Unit Economics

Unit economics ties infrastructure cost to business metrics. Cost per request, cost per user, cost per transaction. This makes infrastructure cost meaningful to business stakeholders.

Unit economics metrics


unit_economics:
  metrics:
    - name: "Cost per Request"
      formula: "total_infrastructure_cost / total_requests"
      target: "Decreasing trend"
      frequency: "Monthly"

    - name: "Cost per Active User"
      formula: "total_infrastructure_cost / active_users"
      target: "Stable or decreasing"
      frequency: "Monthly"

    - name: "Cost per Transaction"
      formula: "total_infrastructure_cost / transactions"
      target: "Decreasing trend"
      frequency: "Monthly"

    - name: "Infrastructure Cost per Revenue Dollar"
      formula: "total_infrastructure_cost / revenue"
      target: "< 10%"
      frequency: "Monthly"

  dashboard:
    panels:
      - title: "Cost per Request (30-day trend)"
        type: "timeseries"
        query: "cost:per_request:daily"

      - title: "Cost by Service"
        type: "pie_chart"
        query: "sum(cost:by_service) by (service)"

      - title: "Cost Efficiency Score"
        type: "gauge"
        query: "actual_cost / optimal_cost"
        target: "< 1.5"

Show cost in business terms

Engineers think in CPU and memory. Business stakeholders think in dollars and users. Translate infrastructure cost into business terms. 'We spend $0.001 per API request' is more meaningful than 'we use 500 CPU cores.'

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.