Apache Kafka
Build event-driven systems that don't lose data. Kafka 4 and KRaft, from your first producer to a streaming platform you'd trust on call. Most Kafka courses still teach you to configure ZooKeeper — Kafka removed it in 4.0. This course teaches the Kafka you'll actually be handed. Built for fresh IT graduates and working engineers alike.
Apache Kafka 4.0 removed ZooKeeper entirely. Not deprecated — removed. Every current cluster runs in KRaft mode, where a quorum of controller nodes manages metadata with the Raft protocol, and there is no in-place upgrade path from a ZooKeeper-based cluster. Yet the overwhelming majority of Kafka courses, tutorials and books still teach ZooKeeper — a component that no longer exists in current Kafka. This course teaches the Kafka you'll actually be handed.
The syllabus is current-generation throughout: KRaft architecture and its own controller and Raft metric namespaces (your ZooKeeper dashboards don't carry over), the new broker-managed consumer rebalance protocol that ends stop-the-world rebalances — including how existing consumer groups migrate — and share groups, the emerging queue-style cooperative consumption that lets consumers scale past the partition count, taught accurately as an evolving capability rather than settled default behaviour. You'll still be able to work on a legacy ZooKeeper cluster — we compare the two architectures clearly — but you'll never be taught to build one.
It is designed for two audiences at once. If you are a fresh IT graduate heading for data or backend engineering, Kafka is infrastructure: organisations that adopt it rarely migrate away, and the skill holds its value. If you are a working engineer — backend, DevOps, SRE or data — the ZooKeeper-to-KRaft transition has left a real gap between what most people were taught and what current clusters run, and this course closes it. The program runs 8 weeks (~65 hours), live online and lab-heavy, closing with an event-driven capstone defended with a documented failure-mode analysis.
Where This Fits in Our Catalogue
- Apache Kafka (this course) — the nervous system: the event backbone the rest of the data platform depends on.
- Apache Spark & PySpark — the processing engine: Kafka feeds the streaming pipelines Spark processes.
- Snowflake Data Warehouse — the governed platform: streaming ingestion lands here.
- MLOps Engineering — event-driven ML systems: models retrained and served on signals, not calendars.
When Kafka Is the Wrong Answer
For low-volume task queues, simple request/response, or workloads a managed queue handles fine, Kafka's operational cost isn't justified — reaching for it reflexively is how platform teams end up on call for infrastructure nobody needed. Knowing that boundary is a senior-engineer signal and a common interview question, and this course teaches it explicitly: you will learn to justify not using Kafka as confidently as using it.
Course at a Glance
- Duration: 8 weeks | ~65 hours | live + hands-on
- Format: Live online · lab-heavy · portfolio of running systems
- Prerequisite: Comfortable programming in Java or Python, basic Linux and networking. No distributed-systems background required.
- Curriculum: 9 modules, from event-driven thinking to cloud deployment and capstone
- Certification: Aligned to the Confluent developer and administrator certification tracks for Apache Kafka
- Curriculum reviewed: July 2026 — Kafka moves through KIPs quickly; this syllabus tracks the current release line
Course Highlights
- Every module leads with what you build and ship, not what you "learn about"
- KRaft from day one — ZooKeeper covered only as legacy context, never as the main path
- The new broker-managed rebalance protocol, including the consumer-group migration path
- Share groups taught honestly as an evolving queue-style capability
- Break-and-recover labs: kill brokers, force elections, watch replication save you
- Schemas as contracts: evolve a topic without breaking live downstream consumers
- Production operations: security, MirrorMaker 2 DR, KRaft-era monitoring, the incidents that actually page you
Hands-on Projects & Portfolio
Stand up a multi-broker cluster, break it deliberately, and recover it.
Idempotent producers, consumer groups and a dead letter topic.
Change a contract without breaking live downstream consumers.
Change data capture from a relational database into Kafka and onward to a warehouse.
Stateful windowed aggregation with exactly-once processing.
TLS, SASL authentication, ACLs and per-client quotas.
Alerting that catches consumer lag before users notice.
End-to-end event-driven system with a documented failure-mode analysis.
Tools & Stack
Apache Kafka 4.x KRaft Kafka Streams Kafka Connect Schema Registry ksqlDB Debezium Avro Protobuf Confluent Cloud AWS MSK Strimzi Kubernetes Docker Prometheus Grafana Java Python Terraform
Certification Alignment
- Developer track: Modules 1–7 cover producers, consumers, schemas, Connect and Streams — the core of the Confluent developer certification for Apache Kafka
- Administrator track: Modules 2, 8 and 9 cover cluster architecture, security, operations and DR — the core of the Confluent administrator certification
- Presented as preparation and alignment — no official partnership or accreditation implied, and no exam outcome guaranteed
Career Outcomes
Kafka is infrastructure, which makes the skill unusually durable — organisations that adopt it rarely migrate away, and engineers who can operate it under load are scarcer than engineers who can write a producer. Backend, DevOps and data engineering are the natural on-ramps, and the ZooKeeper-to-KRaft transition has left a real gap between what most people were taught and what current clusters run. This course targets:
- Data Engineer
- Streaming / Real-Time Data Engineer
- Backend Engineer (event-driven systems)
- Platform Engineer
- Kafka Administrator
- Site Reliability Engineer
- Solutions Architect
- Integration Engineer
Who This Course Is For
- Final-year students and fresh IT graduates targeting data or backend engineering roles
- Java, Python, Go and .NET developers moving into event-driven architecture
- Data engineers adding streaming to batch skills
- DevOps, SRE and platform engineers who have inherited a Kafka cluster
- Architects designing microservice and real-time systems
- Engineers on ZooKeeper-era Kafka planning a KRaft migration
Material Includes
- Class recordings
- Lab guides & cluster starter configs
- Runbooks & incident-response templates
- Certificate
Prerequisites
- Comfortable programming in Java or Python
- Basic Linux and networking
- No distributed-systems background required
Estimated Time: 6 Hours
You ship:- An architecture decision record for a real system: where events beat request/response, where they don't, and where Kafka fits against a traditional broker
- Why request/response breaks down at scale; the log as a primitive; events vs messages vs commands
- Event-driven and event-sourcing patterns
- Where Kafka fits against traditional message brokers; the shape of the problems it actually solves
Estimated Time: 8 Hours
You ship:- A multi-broker KRaft cluster stood up from scratch, then broken deliberately — killed brokers, forced elections — and recovered, with the metadata quorum observed throughout
- Brokers, controllers and the Raft-based metadata quorum; topics, partitions, offsets
- Replication and in-sync replicas; leader election; the commit log and retention; log compaction; tiered storage concepts
- A clear-eyed comparison with the ZooKeeper architecture it replaced — work on legacy clusters without being taught to build one
Estimated Time: 7 Hours
You ship:- A producer hardened against message loss — keys and partitioning chosen deliberately, acknowledgements, idempotence and retries configured with the trade-offs documented
- The send path; keys and partitioning strategy; acknowledgements and durability trade-offs
- Batching, compression and linger; retries and delivery guarantees; idempotent producers
- The configuration choices that quietly decide whether you lose messages
Estimated Time: 8 Hours
You ship:- An order-processing consumer group with commit strategies, a dead letter topic, and a live migration from the classic rebalance protocol to the new broker-managed one
- Offsets and commit strategies; partition assignment; rebalancing and the new broker-managed rebalance protocol — including converting consumer groups between classic and new
- Consumer lag as the metric that matters; error handling and dead letter topics
- Share groups for queue-style cooperative consumption — an evolving capability, taught honestly
Estimated Time: 6 Hours
You ship:- A schema evolution exercise: change a live contract in the Schema Registry without breaking downstream consumers, with compatibility modes chosen and justified
- Avro, Protobuf and JSON Schema; the Schema Registry
- Compatibility modes; evolving a schema without breaking downstream consumers
- The topic as a published API contract between teams
Estimated Time: 7 Hours
You ship:- A change data capture pipeline: Debezium streaming a relational database into Kafka and onward to a warehouse sink, with connector failures handled and offsets managed
- Source and sink connectors; standalone vs distributed workers; converters and transforms
- Change data capture from databases with Debezium; sinking to object storage, warehouses and search
- Connector failure handling and offset management
Estimated Time: 8 Hours
You ship:- A Kafka Streams application with stateful windowed aggregation, joins and exactly-once processing — plus a written engine choice: Streams vs Flink vs Spark for a given workload
- Kafka Streams topologies; KStream and KTable duality; stateful operations and state stores
- Windowing; joins; exactly-once semantics; interactive queries; ksqlDB
- An honest comparison with Apache Flink and Spark Structured Streaming — choose rather than default
Estimated Time: 9 Hours
You ship:- A production-hardened cluster: TLS, SASL, ACLs and quotas; MirrorMaker 2 disaster recovery; and a KRaft-era monitoring stack that catches consumer lag before users notice
- Cluster sizing and partition planning; replication and rack awareness; security (TLS, SASL, ACLs, quotas)
- Multi-cluster and disaster recovery with MirrorMaker 2; monitoring with the KRaft-era metric namespaces — ZooKeeper dashboards don't carry over; Prometheus and Grafana
- Capacity and cost; upgrades and rolling restarts; the incidents that actually page you
Estimated Time: 6 Hours
You ship:- One end-to-end event-driven system — designed, built, deployed and defended with a documented failure-mode analysis — plus a build-vs-buy recommendation across managed offerings
- Managed Kafka on Confluent Cloud, AWS MSK and Azure Event Hubs' Kafka interface
- Kubernetes deployment with Strimzi; infrastructure as code; build-vs-buy economics
- Capstone design, build and defense
- What's App: +91 73-01-11-21-31 for immediate assistance during business hours.
- Email: Send your inquiries to hello@progmatrix.com, and one of our representatives will get back to you as soon as possible.
- Website: You can also use the contact form on our website to send us a message directly.