Stage 7 · Master
Phase 2 — Organization Service
Kubernetes Deployment
Write the manifests that run this image in a real cluster — and deliberately misconfigure one probe first, to see exactly the cascading restart the Health Checks lesson warned about.
Four Manifests, One Directory, One Responsibility Each
services/organization/deploy/k8s/ holds this service's entire Kubernetes footprint: a ConfigMap for non-secret configuration, a Secret for the database URL, a Deployment describing how many replicas run and how they're probed, and a Service exposing them internally. Nothing here is templated or parameterized yet — that gap is exactly what the CI/CD lesson's pipeline will need to solve when it introduces environment-specific values.
apiVersion: v1
kind: ConfigMap
metadata:
name: organization-config
data:
ORGANIZATION_ENV: "production"
ORGANIZATION_PORT: "8080"
ORGANIZATION_LOG_LEVEL: "info"
apiVersion: v1
kind: Secret
metadata:
name: organization-secrets
type: Opaque
stringData:
ORGANIZATION_DATABASE_URL: "postgres://hoa:CHANGE_ME@postgres:5432/hoa_organization?sslmode=require"
This file is a template checked into version control with a deliberately unusable placeholder password — the real value is injected at deploy time by the CI/CD pipeline's secret manager, never committed as an actual credential.
Deliberately Misconfiguring livenessProbe First
The Health Checks lesson's callout predicted a specific failure: pointing livenessProbe at /readyz instead of /healthz. The Deployment below is written with exactly that mistake, on purpose, so the next section can demonstrate the cascading restart directly rather than only describing it.
apiVersion: apps/v1
kind: Deployment
metadata:
name: organization
spec:
replicas: 3
selector:
matchLabels:
app: organization
template:
metadata:
labels:
app: organization
spec:
containers:
- name: organization
image: hoa-platform/organization:dev
ports:
- containerPort: 8080
envFrom:
- configMapRef:
name: organization-config
- secretRef:
name: organization-secrets
livenessProbe:
httpGet:
path: /readyz
port: 8080
periodSeconds: 5
failureThreshold: 2
readinessProbe:
httpGet:
path: /readyz
port: 8080
periodSeconds: 5
Watching the Predicted Cascade Happen, Not Just Reading About It
organization-7d4f9c-a1b2c 1/1 Running 0 3m
organization-7d4f9c-a1b2c 0/1 Running 0 3m (readinessProbe failed: database unreachable)
organization-7d4f9c-a1b2c 0/1 Running 1 3m (livenessProbe failed twice: database unreachable — container restarted)
organization-7d4f9c-a1b2c 0/1 CrashLoopBackOff 2 3m
All three replicas show this pattern nearly simultaneously, because all three share the same database — the exact cascading restart the Health Checks lesson's callout predicted, now reproduced against a real cluster instead of only argued about.
The Fix: livenessProbe Targets /healthz
livenessProbe:
httpGet:
path: /healthz
port: 8080
periodSeconds: 5
failureThreshold: 2
readinessProbe:
httpGet:
path: /readyz
port: 8080
periodSeconds: 5
Only livenessProbe's path changes, from /readyz to /healthz. readinessProbe correctly keeps targeting /readyz — removing a pod from load-balancer rotation during a database outage is exactly the right response; restarting it is not.
apiVersion: v1
kind: Service
metadata:
name: organization
spec:
selector:
app: organization
ports:
- port: 80
targetPort: 8080
Repeating the Same Database Restart Against the Corrected Manifest
Applied exercise
Compute exactly how many failed probes it takes to trigger the original bug
The manifest sets periodSeconds: 5 and failureThreshold: 2 on livenessProbe. Confirm what that specifically means in seconds, then verify it against the reproduction above.
- Calculate, from the manifest's periodSeconds and failureThreshold values alone, the minimum number of seconds of continuous /readyz failure needed before Kubernetes restarts the container under the original (buggy) configuration.
- Re-run the original misconfigured manifest and time, with a stopwatch, how long after stopping Postgres the first container restart actually occurs.
- Explain any discrepancy between your calculation and the observed time (startup probe delay, kubelet sync interval, or similar factors are acceptable answers).
Deliverable
A short note with the calculated threshold, the observed restart time, and an explanation for any gap between them.
Completion checks
- The calculation correctly derives roughly periodSeconds × failureThreshold (10 seconds) as the theoretical minimum.
- The note reasonably accounts for any observed discrepancy rather than treating the two numbers as unrelated.
In the corrected Deployment manifest, why does readinessProbe still target /readyz while livenessProbe targets /healthz?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.