Stage 2 · Tools
DevOps Scripts in Practice
Kubernetes Automation Scripts
kubectl scripting, label management, resource cleanup, and cluster operations.
kubectl Fundamentals
kubectl is the Kubernetes command-line tool. Shell scripts that use kubectl automate deployments, scaling, debugging, and cluster management.
#!/usr/bin/env bash
set -euo pipefail
# Check cluster connection
if ! kubectl cluster-info >/dev/null 2>&1; then
echo "Cannot connect to Kubernetes cluster"
exit 1
fi
# Get pod status
kubectl get pods -n mynamespace -o json | jq -r '.items[] | "\(.metadata.name) \(.status.phase)"'
# Wait for pod to be ready
kubectl wait --for=condition=ready pod -l app=myapp --timeout=300s
# Execute command in pod
kubectl exec -n mynamespace deployment/myapp -- ./health-check.sh
# Get logs
kubectl logs -n mynamespace -l app=myapp --tail=100 -fkubectl wait blocks until a condition is met. -l selects by label. --timeout prevents infinite waits. jq processes JSON output for scripting.
Label Management
#!/usr/bin/env bash
set -euo pipefail
# Add labels to resources
kubectl label nodes node1 environment=production
kubectl label pods mypod tier=frontend
# Update existing label
kubectl label nodes node1 environment=staging --overwrite
# Select by label
kubectl get pods -l app=nginx,environment=production
# Remove label
kubectl label nodes node1 environment-
# List all labels
kubectl get pods --show-labels
# Scale based on labels
REPLICA_COUNT=$(kubectl get pods -l app=myapp --no-headers | wc -l)
echo "Current replicas: $REPLICA_COUNT"Labels are key-value pairs attached to resources. They are used for selection, filtering, and organization. --overwrite updates existing labels. - removes a label.
Resource Cleanup
#!/usr/bin/env bash
set -euo pipefail
NAMESPACE="{1:-default}"
# Delete failed pods
echo "Cleaning up failed pods..."
kubectl get pods -n "$NAMESPACE" --field-selector=status.phase=Failed -o name | \
xargs -r kubectl delete -n "$NAMESPACE"
# Delete completed jobs older than 1 hour
echo "Cleaning up old jobs..."
kubectl get jobs -n "$NAMESPACE" --field-selector=status.successful=1 -o json | \
jq -r '.items[] | select(.status.completionTime < (now - 3600 | todate)) | .metadata.name' | \
xargs -r kubectl delete job -n "$NAMESPACE"
# Delete evicted pods
echo "Cleaning up evicted pods..."
kubectl get pods -n "$NAMESPACE" --field-selector=status.phase=Failed -o json | \
jq -r '.items[] | select(.status.reason == "Evicted") | .metadata.name' | \
xargs -r kubectl delete pod -n "$NAMESPACE"
# Clean up dangling PVCs
echo "Cleaning up unused PVCs..."
kubectl get pvc -n "$NAMESPACE" --no-headers | while read -r name status _; do
if [[ "$status" == "Bound" ]]; then
# Check if any pod is using this PVC
USERS=$(kubectl get pods -n "$NAMESPACE" -o json | \
jq -r ".items[] | .spec.volumes[]? | select(.persistentVolumeClaim.claimName == "$name")" | wc -l)
if (( USERS == 0 )); then
echo "Deleting unbound PVC: $name"
kubectl delete pvc "$name" -n "$NAMESPACE"
fi
fi
doneKubernetes clusters accumulate resources over time. Failed pods, completed jobs, and unused PVCs consume resources. Regular cleanup scripts prevent resource exhaustion.
Rollout Management
#!/usr/bin/env bash
set -euo pipefail
DEPLOYMENT="$1"
NAMESPACE="{2:-default}"
# Start rollout
echo "Starting rollout for $DEPLOYMENT..."
kubectl rollout restart deployment/"$DEPLOYMENT" -n "$NAMESPACE"
# Wait for rollout to complete
echo "Waiting for rollout to complete..."
if kubectl rollout status deployment/"$DEPLOYMENT" -n "$NAMESPACE" --timeout=300s; then
echo "Rollout complete"
else
echo "Rollout failed, initiating rollback..."
kubectl rollout undo deployment/"$DEPLOYMENT" -n "$NAMESPACE"
echo "Rollback initiated"
exit 1
fi
# Show rollout history
kubectl rollout history deployment/"$DEPLOYMENT" -n "$NAMESPACE"rollout restart triggers a new deployment. rollout status waits for completion. If the timeout is exceeded, the script rolls back. This pattern provides zero-downtime deployments with automatic rollback.
Cluster Scripts
#!/usr/bin/env bash
set -euo pipefail
echo "=== Kubernetes Cluster Health ==="
# Node status
echo ""
echo "Nodes:"
kubectl get nodes -o custom-columns=\
'NAME:.metadata.name,STATUS:.status.conditions[-1].type,CPU:.status.capacity.cpu,MEMORY:.status.capacity.memory'
# Pod status by namespace
echo ""
echo "Pods by namespace:"
kubectl get pods --all-namespaces --no-headers | \
awk '{print $1}' | sort | uniq -c | sort -rn
# Failed pods
echo ""
echo "Failed pods:"
FAILED=$(kubectl get pods --all-namespaces --field-selector=status.phase=Failed --no-headers)
if [[ -n "$FAILED" ]]; then
echo "$FAILED"
else
echo "None"
fi
# Resource usage
echo ""
echo "Top pods by CPU:"
kubectl top pods --all-namespaces --sort-by=cpu | head -10
echo ""
echo "Top pods by memory:"
kubectl top pods --all-namespaces --sort-by=memory | head -10This script provides a comprehensive cluster health overview: node status, pod distribution, failures, and resource usage. Run it periodically or on-demand during incidents.
Always use -o json or -o yaml for scripting — not the default table format. JSON is machine-readable and stable across kubectl versions. Process with jq.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.