Stage 7 · Master
Observability
Prometheus Metrics and Health Checks
Every Meridian service already exposes /healthz. This chapter distinguishes that implemented liveness baseline from the planned metrics, readiness, and dependency health surface needed once real tenant-scoped traffic and PostgreSQL arrive.
Implemented Health Baseline
The repository already exposes one health endpoint in every service: GET /healthz returns { service, status }. This is enough for the current scaffolding phase because there are no downstream dependencies, no Postgres pool, no Redis, and no reverse-proxy routes. A passing /healthz today therefore means exactly one thing: the process started and can answer a trivial HTTP request.
Liveness versus Readiness
| Endpoint | What it should answer | Current status |
|---|---|---|
| /healthz or /livez | Is the process alive enough to keep running? | Implemented as /healthz in every service |
| /readyz | Can this instance safely receive production traffic? | Planned for the phase that adds downstream dependencies |
| /metrics | What is the numeric behavior of this process over time? | Planned; not implemented |
Once the gateway proxies to internal services and each service owns real storage, readiness must become stricter than liveness. A billing-service process may be alive while its database pool is exhausted; a gateway process may be alive while all downstream identity requests are failing. Those are readiness failures, not liveness failures. Conflating the two would cause Kubernetes to keep routing traffic to an instance that is running but not useful.
No Prometheus client library, ServiceMonitor, or readiness code is present in meridian today. The remaining sections define the intended surface so the eventual implementation stays aligned with the current service boundaries and Platform Contract.
Planned Metrics Surface
apps/gateway/internal/metrics/metrics.gocreateResponsibility: Register HTTP request duration and dependency outcome metrics for the only public entrypoint.
Why now: The gateway is the correct place to observe public latency and status distribution.
Connects to: Pairs with versioned routes in apps/gateway/internal/router/router.go.
apps/identity-service/internal/health/readiness.gocreateResponsibility: Report readiness based on future database and auth-dependency state rather than on process life alone.
Why now: identity-service will eventually own tenant lookup and membership data, so readiness must reflect that critical path.
Connects to: Complements the existing /healthz handler in cmd/identity-service/main.go.
apps/billing-service/internal/metrics/metrics.gocreateResponsibility: Expose counters and histograms for invoice and payment operations without embedding tenant identifiers in labels.
Why now: Financial traffic needs error-rate and latency visibility, but label cardinality must stay bounded.
Connects to: Will back the future /metrics endpoint for billing-service.
Label Discipline
- Keep labels low-cardinality: service, route template, status class, dependency name, method.
- Do not label metrics with tenant slug, tenant UUID, resident ID, unit ID, invoice ID, or user email.
- Record tenant-specific investigations in logs or traces, not in unbounded Prometheus label sets.
The Decision
Meridian keeps the current /healthz handlers as simple liveness checks during scaffolding, then adds /readyz and /metrics only when real dependencies justify them. The observability model is therefore staged: basic process health now, dependency-aware readiness and bounded Prometheus instrumentation later.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.