Autoscaling

The Right Autoscaling Metric Depends on the Actual Bottleneck, Not a Default

gateway is stateless request routing and JWT verification — CPU-bound work that scales predictably with request rate, making HorizontalPodAutoscaler's default CPU-utilization target a correct fit. organization-api is different: Performance Engineering's connection-pooling incident and the load-testing lesson's 140-VU knee both traced its actual bottleneck to pgxpool connection acquisition, not CPU — a replica can sit at 30% CPU while every request queues on Acquire. Scaling organization-api on CPU would leave it exactly as exposed to pool exhaustion as before Lesson 3's fix; it needs to scale on the same pool-acquired ratio that diagnosed the original incident.

API Gateway already ships a CPU-target HPA, written when the gateway became the platform's single public entry point. That manifest is correct and is not repeated here — CPU utilization genuinely is gateway's bottleneck signal. The question this lesson answers is what to do for a service where the default signal is wrong.

prometheus-adapter Turns a PromQL Expression Into Something an HPA Can Target

The pgxmetrics ratio used to diagnose the original incident — hoa_organization_postgres_pool_acquired_connections / hoa_organization_postgres_pool_max_connections — already exists in Prometheus. prometheus-adapter exposes it to the Kubernetes custom-metrics API under a name an HPA can reference directly, so no new instrumentation is needed in organization-api itself; the metric that diagnosed the incident is the same metric that now prevents it.

rules:
  custom:
    - seriesQuery: 'hoa_organization_postgres_pool_acquired_connections{namespace!="",pod!=""}'
      resources: { overrides: { namespace: { resource: "namespace" }, pod: { resource: "pod" } } }
      name: { as: "organization_postgres_pool_acquired_ratio" }
      metricsQuery: |
        avg(hoa_organization_postgres_pool_acquired_connections{<<.LabelMatchers>>})
        /
        avg(hoa_organization_postgres_pool_max_connections{<<.LabelMatchers>>})

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: organization
  namespace: {{ .Release.Namespace }}
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: organization }
  minReplicas: 2
  maxReplicas: 6 # capped against the shared Postgres budget — see the failure analysis below
  metrics:
    - type: Pods
      pods:
        metric: { name: organization_postgres_pool_acquired_ratio }
        target: { type: AverageValue, averageValue: "700m" }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 15
      policies: [{ type: Pods, value: 2, periodSeconds: 60 }]
    scaleDown:
      stabilizationWindowSeconds: 300
      policies: [{ type: Pods, value: 1, periodSeconds: 180 }]

maxReplicas: 6 Is Not Arbitrary — a Higher Number Silently Reintroduces the Original Incident

The first draft of this HPA set maxReplicas: 20, matching organization's Deployment's own resource headroom with no reference to anything outside the service. Lesson 3 fixed the pool-exhaustion incident by capping organization at MaxConns: 3 per replica specifically because the shared PostgreSQL instance has a 97-connection real budget split across eleven services — that arithmetic did not go away just because scaling is now automatic. At maxReplicas: 20, an HPA doing exactly what it is supposed to do during a real traffic spike would drive organization alone to 60 connections, more than half the platform's entire shared budget, degrading — or reintroducing pool exhaustion for — every other service sharing the same Postgres instance.

Re-run the Load Test That Found the Knee — With Autoscaling Live

A successful run shows organization's replica count rising from 2 toward — but not past — 6 as the k6 ramp crosses the previous knee, the pool-acquired ratio staying below the 0.70 target throughout instead of pegging at 1.0, and the p95 threshold that failed in the load-testing lesson's original run now passing across the full VU range. Autoscaling did not move the platform's real ceiling; it added capacity fast enough, and within a budget-aware limit, to keep the same fixed ceiling from being reached during the same load pattern that previously reached it.