Stage 7 · Master
Phase 19 — Production Deployment
Helm
Convert Lesson 3's raw manifests into a chart, replace the manually scripted kubectl wait with a proper pre-upgrade Helm hook, and parameterize staging and production from one templated source instead of two copy-pasted manifest directories.
Copy-Pasted Manifests Are the Bug Helm Exists to Prevent
Lesson 3's deploy/organization/ directory works for staging. Production needs the same four resources with different replica counts, resource limits, and a different image digest — and the moment those become two separate, hand-maintained YAML directories, a fix applied to one during an incident and forgotten in the other is not a hypothetical risk, it is the default outcome of manual duplication.
replicaCount: 3
image:
api:
repository: ghcr.io/hoa-platform/organization-api
digest: "" # supplied at deploy time by the CI pipeline (Lesson 6); never committed as a placeholder value
migrate:
repository: ghcr.io/hoa-platform/organization-migrate
digest: ""
resources:
requests: { cpu: "100m", memory: "128Mi" }
limits: { memory: "256Mi" }
environment: stagingdigest values are deliberately left empty in the committed file — Lesson 6's promotion pipeline supplies them with --set at deploy time, so an empty value here is a deliberate contract, not an oversight.
replicaCount: 6
resources:
requests: { cpu: "250m", memory: "256Mi" }
limits: { memory: "512Mi" }
environment: productionHelm merges this file over values.yaml rather than replacing it — image repository names, probe paths, and every other unchanged setting are inherited automatically, so production cannot silently diverge on a value nobody meant to change.
Template the Deployment, Do Not Duplicate It
The file in charts/organization/templates is, at this moment, an exact copy of the manifest Lesson 3 applied — which is why the change below can be shown as a diff at all. Four values move into values.yaml and one annotation appears; everything else stays byte-for-byte identical, including the probes, the securityContext and the port names.
metadata:
name: organization-api
- namespace: hoa-staging
+ namespace: {{ .Release.Namespace }}
spec:
- replicas: 3
+ replicas: {{ .Values.replicaCount }}
template:
metadata:
labels: { app: organization-api }
+ annotations:
+ checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
spec:
containers:
- name: api
- image: ghcr.io/hoa-platform/organization-api@${API_IMAGE_DIGEST}
+ image: "{{ .Values.image.api.repository }}@{{ required "image.api.digest is required — supply it with --set at deploy time" .Values.image.api.digest }}"
resources:
- requests: { cpu: "100m", memory: "128Mi" }
- limits: { memory: "256Mi" }
+ requests: { cpu: {{ .Values.resources.requests.cpu }}, memory: {{ .Values.resources.requests.memory }} }
+ limits: { memory: {{ .Values.resources.limits.memory }} }required(...) is the line that earns this template its keep: a missing digest becomes a hard rendering failure rather than a Deployment created with an empty image reference. helm template fails before anything reaches the cluster.
The checksum annotation creates an obligation this lesson has to honour immediately. include (print $.Template.BasePath "/configmap.yaml") is evaluated at render time, so if that file is missing every helm lint, helm template and helm upgrade below fails with a template-not-found error before it can demonstrate anything else. The chart therefore carries a minimal ConfigMap from the start — three settings the Deployment already consumes. The ConfigMaps lesson later expands it and, more importantly, proves what the checksum is actually for.
apiVersion: v1
kind: ConfigMap
metadata:
name: organization-config
namespace: {{ .Release.Namespace }}
data:
LOG_LEVEL: {{ .Values.logLevel | default "info" | quote }}
ORGANIZATION_HTTP_ADDR: ":8081"
ORGANIZATION_INTERNAL_ADDR: ":9090"
ENVIRONMENT: {{ .Values.environment | quote }}Nothing here is sensitive, which is what makes a ConfigMap the right object: every value is safe to read in a kubectl describe output or a support ticket screenshot. The internal listener's address is included deliberately — the probes in the Deployment template target the named internal port, and the two would drift apart if the address were hardcoded in Go and the port number hardcoded in YAML.
Replace the Manually Scripted kubectl wait With a Helm Hook
Lesson 3's ordering guarantee lived in a deploy script's kubectl wait command — correct, but easy to skip if someone runs kubectl apply by hand during an incident. A Helm hook makes the ordering a property of the chart itself: helm upgrade will not proceed to the Deployment until a pre-upgrade hook Job completes successfully, with no separate script required.
metadata:
- name: organization-migrate-${GIT_SHA}
+ name: organization-migrate-{{ .Release.Revision }}
+ annotations:
+ "helm.sh/hook": pre-install,pre-upgrade
+ "helm.sh/hook-weight": "0"
+ "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
spec:
backoffLimit: 2
- ttlSecondsAfterFinished: 3600Three annotations replace a kubectl wait line in a deploy script. hook-delete-policy also makes ttlSecondsAfterFinished redundant: Helm removes the previous revision's Job before creating the next one, so the cleanup the raw manifest delegated to Kubernetes is now the chart's own responsibility.
A pre-upgrade hook Job is deliberately excluded from helm.sh/managed-by ownership the way the Deployment is — this is why hook-delete-policy explicitly manages its own cleanup instead of relying on helm uninstall to remove it automatically.
helm template and helm lint Catch Failures Before the Cluster Ever Sees Them
Error: execution error at (organization/templates/deployment.yaml:20:16):
image.api.digest is required — supply it with --set at deploy timeThis is the exact failure this lesson's required() call was written to produce — a chart that renders successfully without a real digest would be far more dangerous than one that refuses to render at all.
Prove the Hook Actually Blocks a Bad Release
pre-upgrade hooks failed: job failed: BackoffLimitExceeded
Error: UPGRADE FAILED: pre-upgrade hooks failed: job failed: BackoffLimitExceeded
Rolling back organization-api to revision 4Because the Deployment template was never even reached — the hook ran and failed first — the previous revision's Pods, still running the last known-good digest, were never touched. This is the templated equivalent of Lesson 3's 'organization never starts against a broken schema' guarantee, now enforced by Helm itself rather than a script.
Applied exercise
Convert services/gateway-service's manifests into a chart
services/gateway-service has raw manifests from Lesson 3's exercise but no chart, and unlike organization it has no migration Job to hook.
- Scaffold charts/gateway and template its Deployment and Service using .Values for image digest, replica count, and resource sizes.
- Add a required() guard on the image digest value, matching organization's pattern.
- Run helm lint and helm template with the digest omitted, confirming it fails the same way organization's does.
- Explain in one sentence why gateway's chart has no pre-upgrade hook, referencing Lesson 3's exercise answer.
Deliverable
A committed charts/gateway chart, a helm lint pass, a confirmed required() failure without a supplied digest, and a written justification for omitting a migration hook.
Completion checks
- The chart renders correctly with a digest supplied via --set and fails clearly without one.
- No unnecessary pre-upgrade hook Job is present, matching gateway having no schema of its own.
- helm lint reports zero errors against the default values file.
Why does the Deployment template use required(...) around .Values.image.api.digest instead of leaving it optional with a fallback?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.