Stage 7 · Master
Observability
Distributed Tracing Basics
Distributed tracing is not implemented in meridian yet. This chapter specifies how traces should align with the gateway-first request flow, X-Tenant-Id propagation, and the trace_id field already reserved in the API error contract.
Why Tracing Is Planned Early
The repository already has two ingredients that make tracing design worth specifying before instrumentation code exists. First, the gateway is the only public entrypoint, so there is one obvious root for every public trace. Second, libs/platform/errors.APIError already reserves a trace_id field in the failure envelope. Defining the trace model now prevents later disagreement over where traces start, which services own span naming, and how error responses expose correlation identifiers.
There is no OpenTelemetry dependency, trace middleware, or propagated traceparent header in the current repo. The chapter is a specification for the implementation phase, grounded in the existing gateway-first architecture and APIError shape.
Gateway as Trace Root
Every browser request reaches apps/gateway first and only then fans out to internal services. The gateway should therefore start the root span for public traffic, attach tenant slug metadata while the request is still public-facing, resolve the internal tenant UUID, and propagate both tracing context and X-Tenant-Id to the chosen downstream service. Internal services should continue the trace; they should not start unrelated roots for the same request.
trace_id and the Platform Contract
The existing APIError type already commits the wire format to a trace_id string. Once tracing exists, that field should contain the externally useful correlation identifier from the active trace, not a second unrelated request ID. This aligns logs, traces, and error responses: the on-call engineer can copy trace_id from a 4xx or 5xx payload, find the same value in structured logs, and then open the corresponding trace in the collector backend.
Span Boundaries
- Start a root span in gateway for each public HTTP request.
- Create child spans for tenant slug resolution, reverse-proxy forwarding, and each downstream service call.
- Inside identity-service, community-service, and billing-service, create spans around handler execution and major datastore or external-provider operations.
- Use route templates and dependency names for span names; avoid tenant IDs, resident names, or invoice identifiers in span names.
apps/gateway/internal/middleware/tracing.gocreateResponsibility: Start and finish request root spans at the public entrypoint.
Why now: Every public trace needs one authoritative root.
Connects to: Composes with the existing recovery middleware and future request logger.
apps/gateway/internal/proxy/reverse_proxy.gocreateResponsibility: Forward trace context alongside X-Tenant-Id to downstream services.
Why now: Gateway-to-service hops are the first cross-process boundary in the architecture.
Connects to: Uses the same request path that will later proxy versioned routes.
apps/billing-service/internal/http/errors.gocreateResponsibility: Attach the active trace identifier to APIError values returned to callers.
Why now: The Platform Contract already reserves this field; tracing should fill it consistently.
Connects to: Builds on libs/platform/errors.APIError.
The Decision
Meridian plans tracing around the architecture it already has: the gateway starts the public trace, internal services continue it, X-Tenant-Id propagation remains separate from tracing but travels with it, and APIError.trace_id becomes the user-visible correlation handle. The design is specified before instrumentation lands so all later services adopt one trace model rather than four local conventions.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.