Stage 7 · Master

RAG Over Your Ops Knowledge

Chunking Runbooks & Docs

Splitting Markdown, wikis, and postmortems for high-signal retrieval.

7 min readAI for SRE WorkflowsMaster

Why Chunking Matters

LLM context windows are limited and token-based. You cannot send an entire runbook as context for every query. Chunking splits your documents into pieces small enough to fit in the context window while preserving enough context to be useful.

Poor chunking leads to poor retrieval. If a chunk is too small, it lacks context. If too large, it wastes context window on irrelevant content. The sweet spot depends on your document structure.

Chunking Strategies

Strategy	How It Works	Best For
Fixed-size	Split every N characters	Uniform documents, logs
Recursive	Split on paragraph, then sentence, then word	Mixed content
Markdown-aware	Split on headings and sections	Runbooks, docs
Semantic	Split when embedding similarity drops	Narrative content

Markdown-aware chunking


import re

def chunk_markdown(text, max_tokens=500):
    """Split markdown by headings, respecting max token size."""
    sections = re.split(r'(?m)^#{1,3} ', text)
    chunks = []
    current_chunk = ""

    for section in sections:
        # Rough token estimate: 1 token per 4 chars
        section_tokens = len(section) // 4
        current_tokens = len(current_chunk) // 4

        if current_tokens + section_tokens > max_tokens and current_chunk:
            chunks.append(current_chunk.strip())
            current_chunk = section
        else:
            current_chunk += "\n## " + section

    if current_chunk.strip():
        chunks.append(current_chunk.strip())

    return chunks

# Example
runbook = open("redis-failover.md").read()
chunks = chunk_markdown(runbook, max_tokens=500)
print(f"Created {len(chunks)} chunks")

Markdown-aware chunking preserves heading structure so each chunk includes the section header as context.

Chunk Size and Overlap

Chunk size determines how much content each retrieval unit contains. Overlap ensures that content at chunk boundaries is not lost.

Fixed-size chunks with overlap


def chunk_with_overlap(text, chunk_size=1000, overlap=200):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start = end - overlap  # Overlap ensures boundary context
    return chunks

# chunk_size=1000 chars ~ 250 tokens
# overlap=200 chars ~ 50 tokens
# This means each chunk shares 50 tokens with the next chunk

Overlap of 10 to 20 percent of chunk size is typical. Too much overlap wastes storage and creates duplicate results.

Use 300 to 500 token chunks for ops

For runbooks and incident reports, chunks of 300 to 500 tokens work well. They are small enough to fit many in context but large enough to contain a complete procedure step.

Metadata for Filtering

Attach metadata to each chunk so you can filter retrieval by service, severity, date, or document type. This prevents irrelevant chunks from polluting the context.

Chunk with metadata


from datetime import datetime

def chunk_with_metadata(text, source_file, service, doc_type):
    """Chunk text and attach metadata for filtering."""
    raw_chunks = chunk_markdown(text)
    return [
        {
            "text": chunk,
            "metadata": {
                "source": source_file,
                "service": service,
                "doc_type": doc_type,  # "runbook", "postmortem", "doc"
                "chunked_at": datetime.utcnow().isoformat(),
                "token_estimate": len(chunk) // 4,
            }
        }
        for chunk in raw_chunks
    ]

# Example usage
chunks = chunk_with_metadata(
    open("redis-failover.md").read(),
    source_file="runbooks/redis-failover.md",
    service="redis",
    doc_type="runbook"
)

Metadata enables post-retrieval filtering. If a user asks about Redis, you can filter to only Redis-related chunks before semantic search.

Ops-Specific Chunking Patterns

Operational documents have unique structures. Runbooks have numbered steps. Postmortems have timeline, root cause, and action items. Design chunking strategies that respect these structures.

Runbook chunks should keep each numbered step complete with its commands.
Postmortem chunks should separate timeline entries, root cause analysis, and action items.
Log patterns should group related log lines by timestamp or trace ID.
Config files should chunk by section or resource block.

Chunk at natural boundaries

The best chunks follow the natural structure of the document. A chunk that splits a procedure step in half is almost useless. Respect the document structure.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.