Stage 4 · Provision

Terraform State Operations

Drift Response Runbooks

terraform plan -refresh-only, manual changes, ignore_changes, and safe reconciliation workflows — handling drift systematically.

7 min readInfrastructure as CodeProvision

Types of Drift

Type	Cause	Response
Unintentional	Manual console changes, external tools	Revert or import
Intentional	Emergency fixes, auto-scaling	Import into Terraform
Provider-side	Cloud provider updates, deprecations	Update configuration
Tag drift	Organizational tagging policies	ignore_changes

Refresh-only Plans

Refresh-only workflow


$ terraform plan -refresh-only

You can apply this plan to save the new values in terraform.tfstate,
without changing any real infrastructure.

  # aws_instance.web has changed
    ~ tags = {
        - "Name" = "web-server"
        + "Name" = "web-server-prod"
    }

Plan: 0 to add, 0 to change, 0 to destroy.

$ terraform apply -refresh-only

The refresh-only plan updates the state to match reality without proposing resource changes. terraform apply with -refresh-only flag updates the state file. This is useful when external changes are intentional.

Automated drift detection and refresh


name: Drift Detection
on:
  schedule:
    - cron: '0 8 * * 1'

jobs:
  detect:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3

      - name: Check for drift
        run: |
          terraform init
          terraform plan -detailed-exitcode -no-color > plan.out
          EXIT_CODE=$?

          if [ $EXIT_CODE -eq 2 ]; then
            echo "Drift detected"
            # Create issue or send notification
          fi

The weekly job checks for drift. Exit code 2 indicates changes exist. The workflow can create a GitHub issue, send a Slack notification, or trigger a remediation pipeline.

Ignore Changes

ignore_changes patterns


# Ignore tag changes from external systems
resource "aws_instance" "web" {
  lifecycle {
    ignore_changes = [tags["CostCenter"]]
  }
}

# Ignore all tag changes
resource "aws_instance" "web" {
  lifecycle {
    ignore_changes = [tags]
  }
}

# Ignore specific attributes
resource "aws_db_instance" "main" {
  lifecycle {
    ignore_changes = [
      password,
      snapshot_identifier,
      latest_restorable_date,
    ]
  }
}

# Ignore user_data changes
resource "aws_instance" "web" {
  lifecycle {
    ignore_changes = [user_data]
  }
}

ignore_changes tells Terraform to skip certain attributes during diff. This prevents Terraform from reverting changes made by external systems. Use it for tags, passwords, and auto-updated attributes.

Reconciliation Workflows

Safe reconciliation process


#!/bin/bash
# Step 1: Detect drift
terraform plan -detailed-exitcode -no-color > plan.out
EXIT_CODE=$?

if [ $EXIT_CODE -eq 0 ]; then
  echo "No drift detected"
  exit 0
fi

# Step 2: Classify drift
REMOVES=$(grep -c "will be destroyed" plan.out)
CHANGES=$(grep -c "will be updated" plan.out)

# Step 3: Decision
if [ $REMOVES -gt 0 ]; then
  echo "WARNING: Plan includes resource destruction"
  echo "Review manually before applying"
  exit 1
fi

# Step 4: Apply if safe
terraform apply -auto-approve plan.out

The script classifies drift by counting destroys and changes. If resources will be destroyed, it stops for manual review. If only modifications are planned, it applies automatically. This balances safety with automation.

Runbook Templates

Drift response runbook


# Drift Response Runbook

## Step 1: Detect Drift
```bash
terraform plan -detailed-exitcode
```

## Step 2: Identify Cause
- Check CloudTrail for console changes
- Check if external tools modified resources
- Check if organizational policies applied changes

## Step 3: Decide Response
- **Revert**: Apply the Terraform plan to restore desired state
- **Import**: Update Terraform to match the intentional change
- **Ignore**: Add ignore_changes for legitimate external changes

## Step 4: Apply
```bash
terraform apply
# or
terraform apply -refresh-only
```

## Step 5: Document
- Create an incident report
- Update runbook if process changed
- Add ignore_changes if needed

The runbook provides a step-by-step process for handling drift. It ensures consistent response regardless of who handles the incident. The runbook should be versioned and accessible to the team.

Drift Prevention

Restrict IAM permissions — limit who can modify resources outside Terraform.
Use SCPs or Azure Policies — block manual changes at the organization level.
Document the break-glass process — allow manual changes only with approval.
Monitor CloudTrail logs — alert on console API calls that modify managed resources.
Use ignore_changes for legitimate exceptions — auto-scaling, tagging policies.

Run refresh-only after every manual change

If a manual change is intentional, run terraform plan -refresh-only and terraform apply -refresh-only to update the state. This prevents Terraform from reverting the change on the next apply.

Refresh-only does not fix drift

Refresh-only updates the state to match reality. It does not revert the drift. If the drift is unintentional, you need to run terraform apply to revert it. Refresh-only is for acknowledging intentional changes.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.