Rolling Updates

maxSurge and maxUnavailable Decide Whether a Rollout Ever Drops Below Full Capacity

The Deployment written in the Kubernetes lesson has no strategy block, which means it inherits Kubernetes's defaults: maxUnavailable: 25%, maxSurge: 25%. At organization-api's 3 replicas, 25% rounds down to 0 unavailable and up to 1 surge — workable by accident, but nowhere written down, and silently wrong the day someone changes replicas to a number where the same percentages round differently. The fix is to state the intent explicitly: a new pod must be Ready before an old one is removed, so the fleet never has fewer than 3 healthy replicas serving traffic during a deploy.

spec:
  replicas: 3
  minReadySeconds: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          # image, ports, envFrom, probes unchanged from the Kubernetes lesson
          lifecycle:
            preStop:
              sleep: { seconds: 5 }

terminationGracePeriodSeconds Must Cover preStop Plus the App's Own Shutdown Timeout, Not Guess

services/organization/internal/config/config.go already reads a SHUTDOWN_TIMEOUT of 15 seconds — the budget http.Server.Shutdown gets to drain in-flight requests before the process exits. terminationGracePeriodSeconds: 30 is not a round number: 5 seconds for the preStop sleep, plus the 15-second shutdown budget, plus 10 seconds of margin for scheduling jitter — sized against the actual components of shutdown, not chosen because 30 sounded reasonable.

func Run(ctx context.Context, cfg Config, srv *http.Server, ready *atomic.Bool) error {
	ready.Store(true) // flips /readyz to 200 once startup dependencies are confirmed

	stop := make(chan os.Signal, 1)
	signal.Notify(stop, syscall.SIGTERM)

	errCh := make(chan error, 1)
	go func() { errCh <- srv.ListenAndServe() }()

	select {
	case err := <-errCh:
		return err
	case <-stop:
		// Immediately fail /readyz — this pod stops receiving new traffic from
		// the Service well before the process actually exits, because kube-proxy
		// on every node needs a moment to notice the Endpoints removal. The
		// preStop sleep below buys that propagation window.
		ready.Store(false)
	}

	shutdownCtx, cancel := context.WithTimeout(context.Background(), cfg.ShutdownTimeout)
	defer cancel()
	return srv.Shutdown(shutdownCtx) // drains in-flight requests, does not accept new ones
}

A PodDisruptionBudget Protects Availability During Node Drains, Not Just Application Rollouts

maxUnavailable: 0 on the Deployment's rolling-update strategy only governs organization-api's own deploys. A cluster-autoscaler node replacement or a manual kubectl drain during a rollout is a separate, voluntary disruption Kubernetes does not automatically coordinate with the rollout in progress — a PodDisruptionBudget is what stops both events from happening to the same pods at once and dropping below serving capacity.

 spec:
-  minAvailable: 1
+  minAvailable: 2
   selector:
-    matchLabels: { app: gateway }
+    matchLabels: { app: organization-api }

A Broken Image Cannot Silently Finish a Rollout With maxUnavailable: 0

A deploy ships organization-api@sha256:9f2a..., built from a commit that reads a new required config key without a fallback default — the key was added to the Go struct but its ConfigMap entry from Lesson 8 was never updated in the same pull request. Every new pod fails to start, /readyz never turns healthy, and — because maxUnavailable: 0 refuses to remove any working old pod until a new one is confirmed Ready — the rollout stalls instead of completing broken.

Prove Zero Dropped Requests, Not Just a Green Rollout Status

A rollout reported as successful by kubectl is not the same claim as 'no request was dropped while it happened' — the only way to verify the latter is to generate real traffic through the exact window the rollout is running and check its error rate, the same discipline the load-testing lesson established for finding the saturation knee.

A successful verification shows the rollout completing, kubectl get pods transitioning from 3 old pods to 3 new pods with a brief 4-pod surge in between, and the k6 error rate for the entire run staying under the platform's existing 0.1% SLO with no visible spike correlated to the rollout's timestamps — the readiness-gated draining pattern is proven by absence of dropped requests, not by the rollout command's own exit code.