Stage 7 · Master
Phase 2 — Organization Service
Metrics
Instrument every request with Prometheus counters and histograms — using a private registry and route templates, so the same mistake that once caused a real cardinality incident can't happen here.
A Private Registry, Not prometheus.DefaultRegisterer
client_golang's package-level prometheus.MustRegister writes into a shared global registry — convenient for a single-purpose script, but it means two independent tests in the same package that each construct metrics would panic with 'duplicate metrics collector registration attempted' the moment both ran. This service constructs its own prometheus.Registry, owned by a Metrics struct, so it can be built fresh in every test and exactly once in production, with no shared global state anywhere.
Labeling by Route Template, Not Raw Request Path
GET /api/v1/organizations/3f9a1c2e-... and GET /api/v1/organizations/8b2d4f11-... are the same endpoint from every operational standpoint that matters, but they are different strings. Labeling a metric with c.Request.URL.Path directly would create one distinct time series per organization ID ever requested — an unbounded, ever-growing label set that never shrinks even after an organization is archived. c.FullPath() returns Gin's matched route template instead — /api/v1/organizations/:id, one fixed string, regardless of which ID was requested.
package metrics
import (
"strconv"
"time"
"github.com/gin-gonic/gin"
"github.com/prometheus/client_golang/prometheus"
)
// Metrics owns a private *prometheus.Registry rather than registering
// against the package-level default — every test in this course that
// constructs a Metrics gets an isolated registry with no risk of a
// duplicate-registration panic against any other test running in parallel.
type Metrics struct {
registry *prometheus.Registry
requestsTotal *prometheus.CounterVec
requestDuration *prometheus.HistogramVec
}
func New() *Metrics {
registry := prometheus.NewRegistry()
requestsTotal := prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "organization_http_requests_total",
Help: "Total HTTP requests handled, labeled by route template, method, and status code.",
}, []string{"route", "method", "status"})
requestDuration := prometheus.NewHistogramVec(prometheus.HistogramOpts{
Name: "organization_http_request_duration_seconds",
Help: "HTTP request duration in seconds, labeled by route template and method.",
Buckets: prometheus.DefBuckets,
}, []string{"route", "method"})
registry.MustRegister(requestsTotal, requestDuration)
return &Metrics{registry: registry, requestsTotal: requestsTotal, requestDuration: requestDuration}
}
func (m *Metrics) Registry() *prometheus.Registry {
return m.registry
}
// Middleware records exactly one counter increment and one duration
// observation per request. route comes from c.FullPath() — the registered
// route template — never c.Request.URL.Path, which would carry a distinct
// literal value per organization ID and grow this metric's cardinality
// without bound.
func (m *Metrics) Middleware() gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now()
c.Next()
route := c.FullPath()
if route == "" {
route = "unmatched"
}
status := strconv.Itoa(c.Writer.Status())
m.requestsTotal.WithLabelValues(route, c.Request.Method, status).Inc()
m.requestDuration.WithLabelValues(route, c.Request.Method).Observe(time.Since(start).Seconds())
}
}
"unmatched" is a single, fixed fallback label for requests that hit no registered route at all (a typo'd path, a probing bot) — without it, every 404-from-no-route request would otherwise carry an empty-string route label, which is confusing in a dashboard but at least still bounded to one value, never unbounded.
Wiring the Middleware and Exposing /metrics
func New(orgHandler *handler.OrganizationHandler, healthHandler *handler.HealthHandler, m *metrics.Metrics) *gin.Engine {
engine := gin.New()
engine.Use(gin.Recovery())
engine.Use(middleware.ErrorHandler())
engine.Use(m.Middleware())
engine.GET("/healthz", healthHandler.Live)
engine.GET("/readyz", healthHandler.Ready)
engine.GET("/metrics", gin.WrapH(promhttp.HandlerFor(m.Registry(), promhttp.HandlerOpts{})))
v1 := engine.Group("/api/v1")
// organizations group unchanged below this line
gin.WrapH adapts the standard library's http.Handler interface (what promhttp.HandlerFor returns) into the gin.HandlerFunc signature every other route uses — no custom adapter code needed.
repo := repository.NewOrganizationPostgres(pool)
svc := service.NewOrganizationService(repo)
orgHandler := handler.NewOrganizationHandler(svc, validation.New())
healthHandler := handler.NewHealthHandler(pool)
engine := router.New(orgHandler, healthHandler, metrics.New())
Imagine 50,000 organizations created over this service's lifetime. Labeling by raw path would mean organization_http_requests_total accumulates roughly 50,000 distinct time series for GET requests alone — each consuming memory in Prometheus's TSDB indefinitely, even for organizations archived years ago, since Prometheus never knows a label value has become permanently irrelevant. Labeling by route template keeps that same metric at exactly one time series per method, regardless of how many organizations ever exist.
Scraping /metrics by Hand and Confirming Cardinality Stays Flat
organization_http_requests_total{method="GET",route="/api/v1/organizations",status="200"} 1
organization_http_requests_total{method="GET",route="/api/v1/organizations/:id",status="404"} 1
Both requests contributed to exactly two distinct time series — one per distinct route template and status — regardless of the fact that the second request's literal path contained a UUID unique to that one call.
Applied exercise
Reproduce the cardinality problem this design avoids, then confirm the fix
See the raw-path failure mode directly instead of taking the callout's estimate on faith.
- In a disposable local branch of Middleware, temporarily change route := c.FullPath() to route := c.Request.URL.Path.
- Send five requests to five different (even nonexistent) organization IDs under /api/v1/organizations/<id> and then curl /metrics.
- Count how many distinct organization_http_requests_total time series now exist for that endpoint and compare it against the one series produced by the route-template version.
- Revert the change.
Deliverable
A short note with the exact time-series count under both labeling strategies for the same five requests.
Completion checks
- The raw-path version shows five distinct time series; the route-template version shows exactly one.
- Middleware is reverted to use c.FullPath() before finishing.
Why does Middleware label metrics using c.FullPath() instead of c.Request.URL.Path?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.