Stage 6 · Operate
SRE Playbook
Everything an SRE needs — SLOs, error budgets, toil reduction, on-call strategies, and production engineering.
10 modules60 lessons6.5h of reading60/60 live today
Prerequisite
Observability stack.
What this course leaves you with
- Set SLOs and manage error budgets
- Run incidents and write blameless postmortems
- Eliminate toil and operate sustainably on-call
SRESLOIncident ManagementToil
Why this stage matters — Reliability is solid — now turn operations into a reusable platform.
Progress
0 of 60 lessons complete. Progress is stored locally in your browser so you can pick the path back up later.
0%
course completion
01
6 lessons
SLOs & Error Budgets
Quantifying reliability and making it actionable.
- 01Defining SLIsLatency, availability, quality — choosing the right indicators.6 min
- 02Setting SLO TargetsUser expectations, business impact, and the cost of nines.7 min
- 03Error Budget PolicyBurn rate alerting, budget consumption, and freeze thresholds.7 min
- 04Multi-Window Burn RateFast and slow burn detection for paging vs ticketing.6 min
- 05SLO Reporting & ReviewsMonthly reviews, stakeholder communication, and trend analysis.5 min
- 06SLO ToolingPrometheus rules, Grafana dashboards, and Blameless/Protagoras.6 min
02
6 lessons
Toil Reduction Strategies
Identifying, measuring, and eliminating manual repetitive work.
- 01Identifying ToilManual, repetitive, automatable, scales with service — the four criteria.5 min
- 02Runbook AutomationFrom documented procedures to executable automation.7 min
- 03Self-Healing SystemsAuto-remediation, controllers, and the control loop pattern.8 min
- 04Automated Capacity ManagementPredictive scaling, quota management, and resource optimization.7 min
- 05Deployment AutomationProgressive delivery, rollback automation, and deployment safety.7 min
- 06Measuring Toil ReductionToil metrics, time saved, and ROI of automation investments.5 min
03
6 lessons
Incident Response & RCA
Managing incidents effectively and learning from failure.
- 01Incident Command SystemRoles, communication channels, and escalation policies.6 min
- 02Debugging Under PressureStructured debugging, hypothesis-driven investigation.7 min
- 03Stakeholder CommunicationStatus pages, internal comms, and customer-facing updates.5 min
- 04Blameless RCA ProcessTimeline reconstruction, contributing factors, and action items.7 min
- 05Tracking Action ItemsPreventing recurrence, ownership, and closure verification.5 min
- 06Game Days & Fire DrillsPlanned failure injection to validate incident response.6 min
04
6 lessons
On-Call Culture & Sustainability
Making on-call sustainable, fair, and effective.
- 01Rotation DesignFollow-the-sun, primary/secondary, and shadow rotations.6 min
- 02Alert Quality & TuningSignal-to-noise, actionable alerts, and eliminating paging on symptoms.7 min
- 03On-Call CompensationTime off in lieu, monetary compensation, and recognition.5 min
- 04Effective HandoffsShift handoff templates, context transfer, and knowledge sharing.5 min
- 05Mental Health & Burnout PreventionRecognizing burnout, support structures, and team culture.6 min
- 06SRE Career GrowthLadders, competencies, and moving from operator to engineer.5 min
05
6 lessons
Incident Response & On-Call
Operational mechanics for paging, coordinating, communicating, and learning under pressure.
- 01Paging PolicyPage only actionable symptoms using PagerDuty schedules, escalation rules, and ownership maps.6 min
- 02Severity LevelsSEV definitions, customer impact thresholds, escalation triggers, and executive notifications.6 min
- 03Incident Commander PlaybookIC, operations, communications, and scribe roles with clear handoff checkpoints.7 min
- 04Runbook QualityChecklists, kubectl commands, rollback steps, verification queries, and safe automation hooks.7 min
- 05Incident CommunicationsSlack war rooms, status pages, customer updates, and timestamped decision logs.6 min
- 06Postmortem FacilitationBlameless timelines, contributing factors, action-item owners, and recurrence metrics.7 min
06
6 lessons
Capacity & Performance Engineering
Forecasting demand, validating limits, and tuning systems before users notice pain.
- 01Load Testing Strategyk6, Locust, realistic traffic mixes, ramp profiles, and failure criteria.8 min
- 02Capacity PlanningGrowth models, headroom targets, quota limits, and seasonality forecasting.7 min
- 03Autoscaling StrategyHPA targets, KEDA triggers, predictive scaling, cooldowns, and scale-to-zero risks.7 min
- 04Performance ProfilingFlame graphs, pprof, perf, and tracing slow paths across service boundaries.8 min
- 05Cost & EfficiencyRightsizing, bin packing, spot capacity, reservation planning, and unit economics.6 min
- 06USE & RED MethodsUtilization, saturation, errors, rate, duration, and dashboards that expose bottlenecks.6 min
07
6 lessons
Alerting Hygiene & Paging Design
Actionable alerts, paging policy, routing rules, and noise reduction for sustainable operations.
- 01Actionable AlertsPage on user-impacting symptoms using Prometheus alerts, runbook links, and ownership labels.6 min
- 02Paging PhilosophyTicket versus page decisions, urgency thresholds, escalation paths, and sleep-impact tradeoffs.6 min
- 03Alert RoutingAlertmanager routes, PagerDuty services, team ownership maps, and maintenance silences.7 min
- 04Noise ReductionDeduplication, grouping, inhibition rules, flapping detection, and stale alert cleanup.7 min
- 05Alert ReviewWeekly alert audits, page volume metrics, missed pages, and tuning action queues.5 min
- 06Handoff ReadinessShift summaries, active incidents, risky deploys, ownership changes, and follow-the-sun transfer notes.6 min
08
6 lessons
Blameless Postmortems & Learning
Incident timelines, contributing factors, corrective actions, and organizational learning practices.
- 01Timeline ReconstructionChat logs, deploy events, metrics annotations, alert timestamps, and decision records.6 min
- 02Contributing FactorsFive whys, human factors, latent conditions, safeguards, and systems thinking.7 min
- 03Writing PostmortemsImpact summaries, detection gaps, response analysis, root causes, and customer-visible language.7 min
- 04Action Item QualitySMART actions, durable owners, due dates, prevention controls, and verification evidence.6 min
- 05Learning ReviewsFacilitated reviews, psychological safety, near misses, and sharing lessons across teams.7 min
- 06Recurrence TrackingIncident tags, repeat failure modes, action aging dashboards, and quarterly reliability themes.6 min
09
6 lessons
Reliability Testing & Chaos Engineering
Game days, controlled failure injection, resilience experiments, and recovery validation drills.
- 01Experiment DesignHypotheses, steady-state metrics, blast radius controls, abort criteria, and observer roles.7 min
- 02Chaos ToolingLitmusChaos, Chaos Mesh, Gremlin, fault injection, and Kubernetes failure scenarios.8 min
- 03Game Day PlanningScenario scripts, participant roles, communication channels, customer-impact guardrails, and debriefs.7 min
- 04Dependency Failure TestingTimeouts, retries, circuit breakers, DNS failures, and downstream saturation experiments.8 min
- 05Rollback DrillsCanary aborts, feature flag disables, database rollback plans, and deployment verification checks.7 min
- 06Resilience ScorecardsExperiment results, control gaps, service maturity ratings, and reliability roadmap inputs.6 min
10
6 lessons
Disaster Recovery & Continuity
Recovery objectives, backup validation, failover plans, and business continuity exercises.
- 01RTO & RPO TargetsService tiers, data-loss tolerance, recovery time objectives, and stakeholder signoff.6 min
- 02Backup ValidationRestore tests, checksum verification, backup retention, immutable storage, and audit evidence.7 min
- 03Failover ArchitectureActive-active, active-passive, DNS failover, traffic managers, and regional dependencies.8 min
- 04DR RunbooksDecision trees, command checklists, access prerequisites, communications, and rollback criteria.7 min
- 05Continuity ExercisesTabletop scenarios, regional evacuation drills, supplier failures, and executive escalation paths.7 min
- 06Recovery MetricsActual recovery time, data freshness, manual steps, failed controls, and remediation tracking.6 min