Stage 7 · Master
Phase 1 — Project Foundation
Kafka Setup
Bring up a single-broker Kafka in KRaft mode locally, without a Zookeeper container you would never actually run alongside it in production.
Why KRaft Mode, Not a Second Zookeeper Container
For most of Kafka's history, running it locally meant running Zookeeper alongside it — a second stateful system solely to coordinate the first one. Since Kafka's KRaft mode (Kafka Raft metadata mode) reached production readiness, Kafka coordinates its own metadata internally, and Zookeeper is no longer required at all. Provisioning a Zookeeper container in this compose file would mean teaching, and running, an operational dependency this platform's production Kafka deployment will not use either — the local environment should mirror the real one, not a historical one.
The KRaft Environment Variable Contract
A single-broker KRaft node needs to know its own node ID, which roles it plays (in a single-node setup, both broker and controller), who the controller quorum voters are (itself, in this case), which listener handles controller traffic versus client traffic, and that plaintext (non-TLS) connections are acceptable for local development. Every one of these is a required environment variable, not a default — a KRaft broker that is missing one of them refuses to start rather than guessing.
volumes:
pg-data:
redis-data:
+ kafka-data:
services:
+ kafka:
+ image: bitnami/kafka:3.7
+ restart: unless-stopped
+ environment:
+ KAFKA_CFG_NODE_ID: "1"
+ KAFKA_CFG_PROCESS_ROLES: broker,controller
+ KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093
+ KAFKA_CFG_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
+ KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
+ KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
+ ALLOW_PLAINTEXT_LISTENER: "yes"
+ ports:
+ - "9092:9092"
+ volumes:
+ - kafka-data:/bitnami/kafka
+ networks:
+ - hoa-net
PostgreSQL, Redis, Adminer, and the shared network are unchanged. The diff isolates Kafka's KRaft contract instead of printing the complete Compose file for a third time.
Proving a Broker Is Not the Same as Proving a Topic
A container reporting 'Up' only proves the Kafka process started — it says nothing about whether the broker actually finished forming its single-node quorum and can serve client requests. This lesson's verification therefore does not stop at docker compose ps; it creates a real topic, lists it back, and only then trusts the broker.
Bitnami's Kafka image refuses to start with an unauthenticated, unencrypted (plaintext) listener unless this variable explicitly opts in, to avoid anyone accidentally shipping that configuration to production. Omitting it produces a startup failure citing the plaintext listener as unsafe — this platform's real deployment will use TLS and SASL, configured in the Kubernetes Deployment and CI/CD lessons of Phase 2, and will not carry this variable at all.
Confirming the Whole Compose File Together
Applied exercise
Reproduce the plaintext-listener failure this callout describes
The callout states a consequence without showing it. See the actual failure before trusting the explanation.
- Temporarily remove the ALLOW_PLAINTEXT_LISTENER line from the kafka service definition.
- Run docker compose up kafka (without -d, so logs stream to your terminal) and read the startup failure.
- Record the exact log line that names the plaintext listener as the problem.
- Restore ALLOW_PLAINTEXT_LISTENER: "yes" and confirm the broker starts cleanly again.
Deliverable
A short note quoting the exact failure line from step 2, and confirmation the broker recovers once the variable is restored.
Completion checks
- The failure message specifically references the plaintext or unauthenticated listener, not an unrelated startup error.
- docker-compose.yml matches this lesson's version exactly by the end of the exercise.
Why does this lesson's Kafka configuration skip Zookeeper entirely?
Mark this lesson complete to store local progress and unlock a cleaner resume path the next time you visit.