Stage 7 · Master
Background Processing
Worker Pools in Go
Meridian's background-processing design centers on billing-service and a planned second cmd/worker binary inside the same module.
Why billing-service Needs Workers
There is no worker process in meridian today, and there is no standalone task-worker service in the real architecture. When Meridian eventually needs asynchronous work, billing-service is the natural first owner. Payment-webhook reconciliation, recurring dues invoice generation, and receipt rendering are all workflows that can outlive the latency budget of a synchronous HTTP request while still belonging squarely to the billing bounded context.
The recommended design is a second binary inside apps/billing-service, for example apps/billing-service/cmd/worker/main.go. This stays consistent with the cmd/<service>/ convention established earlier: one module can legitimately ship more than one binary when the bounded context owns both HTTP and background execution.
Where the Worker Binary Lives
apps/billing-service/cmd/worker/main.gocreateResponsibility: Starts the billing worker process, loads config, wires queue adapters, and runs the worker pool until shutdown.
Why now: A second binary keeps background execution separate from the HTTP server without inventing a separate microservice boundary.
Connects to: Extends the cmd/<service>/ pattern already used by apps/billing-service/cmd/billing-service/main.go.
apps/billing-service/internal/worker/pool.gocreateResponsibility: Defines worker goroutines, concurrency settings, and the dispatch loop for billing jobs.
Why now: Concurrency policy should be centralized instead of spread across ad hoc goroutine launches in job handlers.
Connects to: Uses internal/config once billing-service gains Redis or database queue configuration.
apps/billing-service/internal/jobs/job.gocreateResponsibility: Defines the tenant-scoped job payloads processed by the pool.
Why now: Background jobs are part of the billing-service contract and need explicit schema, versioning, and ownership.
Connects to: Supports future invoice, payment, and receipt workflows owned by billing-service.
Pool Shape and Job Payload
The first jobs do not need a large abstraction hierarchy. A recurring invoice generator can accept a billing period plus tenant_id. A payment-webhook reconciliation job can carry a provider event ID plus tenant_id. A receipt-render job can carry invoice_id or payment_id plus tenant_id. The worker pool should execute these uniformly while the job handler applies billing-specific logic per type.
Tenant Awareness inside Workers
Background work is not exempt from multi-tenancy. A billing job that omits tenant_id would force the worker to infer scope from global IDs or from an extra lookup, both of which weaken isolation guarantees. tenant_id must be part of the queue message, part of the logging context, part of cache keys touched during execution, and part of every SQL predicate once billing-service owns a database schema. The worker should be able to fail fast if tenant context is missing.
The Decision
- Meridian does not introduce a separate worker microservice; the first worker belongs inside apps/billing-service as a second cmd/ binary.
- Worker pools are justified first for billing workflows such as invoice generation, payment reconciliation, and receipt rendering.
- Every job payload carries tenant_id explicitly and logs with tenant-aware context.
- The pool should stay small and mechanical; billing-specific behavior belongs in job handlers, not in concurrency plumbing.
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.