Stage 3 · Build

Observability & Networking

OpenTelemetry Go

Tracing, metrics, logs — the OTel SDK and auto-instrumentation for Go services.

8 min readGo for SREBuild

OpenTelemetry Overview

OpenTelemetry is the standard for observability instrumentation. It provides unified APIs for traces, metrics, and logs. The Go SDK lets you instrument your service once and export to any backend — Jaeger, Prometheus, Datadog, or Grafana.

Gootel-sdk-setup.go

26 linesLn 1, Col 1Go

resource.New creates a resource describing your service. TracerProvider manages span creation and export. WithBatcher buffers spans for efficient export. SetTracerProvider registers the provider globally.

Distributed Tracing

Traces follow requests across service boundaries. A trace consists of spans, each representing a unit of work. Spans capture timing, status, and context.

Gocreating-spans.go

29 linesLn 1, Col 1Go

tracer.Start creates a child span. defer span.End ensures the span is closed. Attributes add context to the span. RecordError captures errors. SetStatus marks the span as failed.

OTel Metrics

OTel metrics provide a unified API for counters, gauges, and histograms. The API is similar to Prometheus but with the OTel SDK's resource and instrumentation scope management.

Gootel-metrics.go

35 linesLn 1, Col 1Go

OTel metrics use callbacks for gauges — the SDK calls your function to get the current value. Counters and histograms are recorded explicitly. The SDK handles aggregation and export.

Context Propagation

Context propagation passes trace context across service boundaries via HTTP headers. This connects spans into a distributed trace. OTel handles this automatically for HTTP clients and servers.

Gohttp-propagation.go

25 linesLn 1, Col 1Go

otelhttp.NewHandler wraps your HTTP handler to extract trace context from incoming requests. otelhttp.NewTransport wraps your HTTP client to inject trace context into outgoing requests. This connects your traces across services.

Auto-Instrumentation

The OTel Go contrib packages auto-instrument common libraries: net/http, gRPC, database/sql, and more. Wrap your clients and handlers to get instrumentation without changing code.

Goauto-instrumentation-packages.go

28 linesLn 1, Col 1Go

Auto-instrumentation wraps standard library types to create spans and metrics. otelhttp instruments HTTP requests. otelgrpc instruments gRPC calls. otelsql instruments database queries. All propagation is handled automatically.

OTel Exporters

Exporters send telemetry data to backends. OTel supports multiple backends through the exporter interface. Choose based on your observability stack.

Gocommon-exporters.go

23 linesLn 1, Col 1Go

OTLP is the standard protocol. Use it with an OTel Collector for flexible routing. Jaeger and Zipkin exporters send directly to those backends. Prometheus exporter bridges OTel metrics to Prometheus format.

Use the OTel Collector

The OTel Collector receives telemetry via OTLP and routes it to multiple backends. It handles batching, retrying, and sampling. Deploy it as a sidecar or DaemonSet for efficient telemetry routing.

Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.