Stage 7 · Master

LLM Foundations for Operators

Prompting for Ops Tasks

System prompts, few-shot examples, and structured JSON output.

7 min readAI for SRE WorkflowsMaster

Anatomy of an Ops Prompt

A prompt has three parts: a system message that defines the model's role, context that provides the relevant information, and the user's actual question. For operational tasks, getting all three right is critical.

Basic prompt structure


response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[
        {
            "role": "system",
            "content": "You are an SRE assistant. Answer based ONLY on the provided context."
        },
        {
            "role": "user",
            "content": """Context from runbook:
            Redis Failover Procedure:
            1. Check current master with: redis-cli -h redis-prod info replication
            2. If master is down, promote replica: redis-cli -h replica-1 cluster failover

            Question: Redis is unreachable. What do I do?"""
        }
    ]
)

The system prompt sets boundaries. The context gives the model something to ground its answer in. The question directs the output.

System Prompts

System prompts define the model's behavior, role, and constraints. For SRE tasks, always include explicit instructions about what the model should and should not do.

System prompt for incident triage


SYSTEM_PROMPT = """You are an incident triage assistant for a cloud infrastructure team.

Rules:
- Classify alerts as P1 (critical), P2 (high), P3 (medium), or P4 (low)
- You MUST cite the source of your classification
- If uncertain, say "I cannot determine severity" rather than guessing
- Never suggest running destructive commands without explicit warnings
- Always ask for more context if the alert description is ambiguous

Output format: JSON with severity, reasoning, and recommended next steps."""

Strong system prompts include role definition, behavioral rules, uncertainty handling, and output format specification.

Never trust AI severity classifications alone

An LLM may classify a database corruption alert as P3 because it does not understand the downstream impact. Always have a human validate severity for P1 and P2 classifications.

Few-Shot Examples

Few-shot prompting provides examples of the desired input-output pattern. This is one of the most effective techniques for operational tasks because it teaches the model the exact format you need.

Few-shot alert classification


messages = [
    {"role": "system", "content": "Classify alerts by severity."},
    {
        "role": "user",
        "content": "Alert: CPU usage 95% on web-server-01 for 10 minutes"
    },
    {
        "role": "assistant",
        "content": '{"severity": "P3", "reason": "High CPU but no user impact reported yet", "action": "Monitor for escalation"}'
    },
    {
        "role": "user",
        "content": "Alert: Database primary replica lag exceeded 30 seconds"
    },
    {
        "role": "assistant",
        "content": '{"severity": "P2", "reason": "Replication lag risks data inconsistency and read failures", "action": "Check replication health, consider failover"}'
    },
    {
        "role": "user",
        "content": "Alert: Redis connection pool exhausted, 100% error rate on checkout service"
    }
]

Each example pair teaches the model the format and reasoning pattern. The model generalizes from these examples to classify the new alert.

Structured JSON Output

For operational pipelines, you need machine-parseable output. Explicitly requesting JSON with a schema description ensures the model produces consistent, structured responses that your automation can consume.

Forcing structured JSON output


response = openai.ChatCompletion.create(
    model="gpt-4",
    temperature=0.0,
    messages=[
        {
            "role": "system",
            "content": """Respond ONLY with valid JSON matching this schema:
{
    "severity": "P1|P2|P3|P4",
    "confidence": 0.0-1.0,
    "reasoning": "string",
    "affected_services": ["string"],
    "next_steps": ["string"]
}"""
        },
        {"role": "user", "content": alert_text}
    ]
)

Using temperature 0 and an explicit JSON schema in the system prompt dramatically improves output consistency. Always validate the output before using it in automation.

Reusable Prompt Templates

Build a library of prompt templates for common operational tasks. Version them in Git alongside your runbooks. Treat prompts as code — they affect your production behavior.

Prompt template registry


PROMPT_TEMPLATES = {
    "alert_triage": {
        "system": "You are an alert triage assistant...",
        "user": "Classify this alert: {alert_text}

Context: {context}",
        "model": "gpt-4",
        "temperature": 0.0,
    },
    "runbook_lookup": {
        "system": "Answer based ONLY on the provided runbook...",
        "user": "Runbook:
{runbook_content}

Question: {question}",
        "model": "gpt-4",
        "temperature": 0.0,
    },
    "incident_summary": {
        "system": "Summarize this incident for a status page...",
        "user": "Incident channel transcript:
{transcript}",
        "model": "gpt-4-turbo",
        "temperature": 0.3,
    },
}

def get_prompt(task_name, **kwargs):
    template = PROMPT_TEMPLATES[task_name]
    return {
        "model": template["model"],
        "temperature": template["temperature"],
        "messages": [
            {"role": "system", "content": template["system"]},
            {"role": "user", "content": template["user"].format(**kwargs)},
        ],
    }

Centralizing prompt templates makes them versionable, testable, and easy to update without changing application code.

Version your prompts

Store prompt templates in a dedicated directory with semantic versioning. When you update a prompt, run your eval suite to catch regressions before deploying.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.