Stage 5 · Platform
Progressive Delivery Controllers
Rollout Analysis
Designing AnalysisTemplates with success conditions, failure limits, intervals, and metric providers.
Analysis Concept
AnalysisTemplates define success and failure criteria for progressive delivery. They query metrics from external providers (Prometheus, Datadog, CloudWatch) and evaluate conditions. If conditions pass, the rollout progresses. If they fail, the rollout is aborted.
Analysis is the core of automated canary analysis. Without it, canary deployments require manual verification. With it, canaries are automatically promoted or rolled back based on measurable criteria.
Analysis Templates
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: canary-analysis
namespace: production
spec:
args:
- name: service-name
value: myapp
- name: namespace
value: production
metrics:
- name: success-rate
interval: 2m
count: 5
successCondition: result[0] >= 0.99
failureLimit: 3
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
sum(rate(http_requests_total{
service="{{args.service-name}}",
namespace="{{args.namespace}}",
status!~"5.."
}[2m])) /
sum(rate(http_requests_total{
service="{{args.service-name}}",
namespace="{{args.namespace}}"
}[2m]))
- name: latency-p95
interval: 2m
count: 5
successCondition: result[0] <= 500
failureLimit: 3
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
histogram_quantile(0.95,
sum(rate(http_request_duration_seconds_bucket{
service="{{args.service-name}}",
namespace="{{args.namespace}}"
}[2m])) by (le)
) * 1000
- name: error-rate
interval: 1m
count: 3
successCondition: result[0] <= 0.01
failureLimit: 2
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
sum(rate(http_requests_total{
service="{{args.service-name}}",
namespace="{{args.namespace}}",
status=~"5.."
}[1m])) /
sum(rate(http_requests_total{
service="{{args.service-name}}",
namespace="{{args.namespace}}"
}[1m]))Each metric runs independently. successCondition is a boolean expression evaluated against the query result. failureLimit allows some failures before aborting. count determines how many measurements to take.
Prometheus Provider
metrics:
# Range vector query
- name: cpu-usage
interval: 1m
count: 10
successCondition: result[0] <= 80
provider:
prometheus:
address: http://prometheus:9090
query: |
100 * (
sum(rate(container_cpu_usage_seconds_total{
namespace="{{args.namespace}}",
pod=~"myapp-.*"
}[1m])) /
sum(kube_pod_container_resource_limits{
namespace="{{args.namespace}}",
pod=~"myapp-.*",
resource="cpu"
})
)
# Instant vector query
- name: memory-usage
interval: 1m
count: 5
successCondition: result[0] <= 85
provider:
prometheus:
address: http://prometheus:9090
query: |
100 * (
container_memory_working_set_bytes{
namespace="{{args.namespace}}",
pod=~"myapp-canary-.*"
} /
kube_pod_container_resource_limits{
namespace="{{args.namespace}}",
pod=~"myapp-canary-.*",
resource="memory"
}
)Prometheus queries can use range vectors (rate, increase) or instant vectors. Range queries evaluate over a time window, providing more stable results. Instant queries evaluate at a single point in time.
Datadog Provider
metrics:
- name: datadog-success-rate
interval: 2m
count: 5
successCondition: result[0] >= 99
failureLimit: 3
provider:
datadog:
site: datadoghq.com
apiKey:
secretName: datadog-api-key
secretKey: api-key
appKey:
secretName: datadog-app-key
secretKey: app-key
query: |
sum:http.requests{service:myapp,status_code:2xx}.as_rate() /
sum:http.requests{service:myapp}.as_rate() * 100
- name: datadog-latency
interval: 2m
count: 5
successCondition: result[0] <= 500
failureLimit: 3
provider:
datadog:
site: datadoghq.com
apiKey:
secretName: datadog-api-key
secretKey: api-key
appKey:
secretName: datadog-app-key
secretKey: app-key
query: |
avg:http.request.duration{service:myapp}.as_count()The Datadog provider uses Datadog API keys stored as Kubernetes Secrets. The query uses Datadog's query syntax. The result is a scalar value evaluated against the success condition.
CloudWatch Provider
metrics:
- name: cloudwatch-5xx-rate
interval: 2m
count: 5
successCondition: result[0] <= 1
failureLimit: 3
provider:
cloudwatch:
region: us-east-1
namespace: AWS/ApplicationELB
metricDataQueries:
- id: error_rate
expression: "errors / total * 100"
label: Error Rate
- id: errors
metricStat:
metric:
metricName: HTTPCode_Target_5XX_Count
dimensions:
- name: LoadBalancer
value: app/myapp-alb/abc123
period: 120
stat: Sum
- id: total
metricStat:
metric:
metricName: RequestCount
dimensions:
- name: LoadBalancer
value: app/myapp-alb/abc123
period: 120
stat: SumCloudWatch provider uses metric data queries with expressions. The expression calculates the 5xx error rate as a percentage. The ALB dimensions identify the specific load balancer.
Custom Providers
metrics:
- name: custom-check
interval: 1m
count: 3
successCondition: result[0] == true
failureLimit: 2
provider:
webhook:
url: http://analysis-webhook:8080/check
timeout: 10s
metadata:
type: json
method: POST
body: |
{
"service": "{{args.service-name}}",
"namespace": "{{args.namespace}}",
"canary_pod": "{{args.canary-pod}}"
}The webhook provider calls a custom API for analysis. The response must include a result field that is evaluated against the successCondition. Use this for custom business logic that cannot be expressed as a metric query.
Define args in AnalysisTemplates to make them reusable across services. Pass service name, namespace, and thresholds as arguments. This avoids duplicating templates for each service.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.