Stage 7 · Master
Phase 16 — Kafka Integration
Event Versioning
Draw the line between a change a tolerant reader absorbs for free and a change that requires a second topic and a dual-publish window, and give the consumer a safety net for the difference.
encoding/json Already Makes This Consumer a Tolerant Reader
paymentCompletedEnvelope (Lesson 3) decodes JSON into a struct with named fields. If payment-service adds a new field to CompletedPayload tomorrow — a discount_code, say — this consumer's json.Unmarshal call silently ignores it; the struct simply has no field to put it in. That is not an accident to guard against, it is the entire point of decoding into a known struct rather than a generic map: additive changes cost this consumer nothing.
Renaming or Retyping a Field Is Not Additive
| Change | Compatible? | Why |
|---|---|---|
| Add a new optional field to Payload | Yes | Old consumers ignore it via json.Unmarshal; new consumers see it |
| Add a new event_type value on the same topic | Yes, if consumers switch on known types and ignore unknown ones | Requires the consumer to already tolerate unrecognized event_type — this platform's does not yet, see below |
| Rename provider_reference to payment_reference | No | Old field name simply stops arriving; consumers reading provider_reference silently get zero value forever |
| Change amount_paise from an integer to a nested {value, currency} object | No | json.Unmarshal fails outright — the type no longer matches the struct field |
| Remove a field consumers depend on | No | Same failure mode as a rename |
Lesson 3's Handler is wired specifically to payment.completed and assumes every message on that topic is a completed-payment event. Before a second event_type could safely share this topic, the handler would need an explicit switch on event_type with a default case that skips unrecognized values rather than failing to parse them — that change is out of scope here and left as a prerequisite for anyone extending this topic to more than one event type.
payment.completed.v2 Coexists With payment.completed During Migration
A genuinely breaking change — the amount_paise-to-nested-object example above — does not get retrofitted onto payment.completed. It gets its own topic, payment.completed.v2, and payment-service's relay publishes to both topics for a fixed migration window while every consumer moves over at its own pace.
The relay change to publish twice is small — it calls the same WritePaymentCompleted-style function twice, once per topic, inside the same outbox-drain transaction, so both publishes share the exact at-least-once and ordering guarantees Lesson 2 already established for one.
The Consumer Checks event_version Itself, Rather Than Trusting the Topic Name Alone
Relying purely on "this consumer only reads payment.completed, and payment.completed is always v1 shape" is an assumption baked into a subscription, not a fact enforced anywhere. Lesson 5 already gave the handler a place to reject anything it can't safely process: this lesson adds an explicit event_version check right next to the unmarshal, so a v2-shaped message accidentally published to the v1 topic during a migration mistake fails loudly and immediately as a poison event, rather than partially decoding into a struct that silently drops the fields that moved.
var env paymentCompletedEnvelope
if err := json.Unmarshal(raw, &env); err != nil {
return errPoisonEvent(err)
}
if env.EventVersion != 1 {
return errPoisonEvent(fmt.Errorf("unsupported event_version %d on payment.completed", env.EventVersion))
}
Confirm an Additive Field Is Ignored and a Version Mismatch Is Rejected
Applied exercise
Write the migration checklist for renaming provider_reference
Renaming provider_reference to payment_reference is flagged above as a breaking change. Write the concrete steps this platform's conventions actually require to ship it safely.
- List the exact new topic name this change requires, following the payment.completed.v2 convention shown above.
- Describe the relay-side change needed to dual-publish, referencing Lesson 2's OutboxWriter/Relay.
- Describe the consumer-side change needed, including how event_version distinguishes the two shapes at the point of the existing check added in this lesson.
- State the condition that must be true before the dual-publish window can end and payment.completed.v1 publishing can stop.
Deliverable
A short, ordered checklist (5-8 steps) covering producer, consumer, and cutover criteria.
Completion checks
- The checklist correctly separates producer-side dual-publish from consumer-side version handling as distinct steps.
- The cutover criterion is concrete (e.g., 'all known consumer groups have deployed v2 support and consumer lag on v1 is confirmed zero for N days'), not just 'wait a while.'
Why does adding a new optional field to Payload require no consumer-side change, while renaming provider_reference does?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.