INTERMEDIATE LEVELKAFKA
Event-Driven Architecture & Distributed Systems
Apache Kafka and Event-Driven Systems: Building Real-Time Streaming Pipelines
Master event-driven architecture, distributed commit logs, Kafka internals, schema governance, Apache Flink stream processing, and end-to-end exactly-once pipelines.
Est. Duration
~14 Hours
Curriculum
10 Chapters • 40 Lessons
Capstones
7 Global Arena
Total Bounty
+845 XP
Your Progress0 of 40 Lessons (0%)
Start Here:Chapter 1.1 — Request-Response vs Event-Driven Architecture: When to Decouple Microservices
Curriculum Syllabus
10 Chapters • 40 Lessons
1.1Request-Response vs Event-Driven Architecture: When to Decouple Microservices
7 min+15 XP
1.2Events vs Commands vs Queries: Domain Modeling in Event-Driven Systems
7 min+15 XP
1.3The Dual-Write Problem in Distributed Systems and How to Solve It
7 min+15 XP
1.4The Transactional Outbox and Inbox Patterns: Reliable Event Publishing⚔️ Landmark Capstone
8 min+50 XP
2.1Traditional Message Queues: How RabbitMQ and AWS SQS Manage State
8 min+15 XP
2.2What is a Distributed Commit Log? How Apache Kafka and Redpanda Work
8 min+15 XP
2.3Pull vs Push Consumption in Messaging: Backpressure and Flow Control
8 min+15 XP
2.4Message Delivery Guarantees: At-Most-Once, At-Least-Once, and Exactly-Once
8 min+15 XP
3.1Apache Kafka Architecture: Topics, Partitions, Clusters, and Replication
8 min+15 XP
3.2Kafka Storage Internals: Anatomy of Log Segments, .index, and .timeindex Files⚔️ Landmark Capstone
8 min+50 XP
3.3Why Kafka is Fast: Sequential Disk I/O, Linux PageCache, and Zero-Copy
8 min+15 XP
3.4Kafka KRaft Mode vs ZooKeeper: Architecture, Quorums, and Performance
8 min+15 XP
4.1Kafka Producer Architecture: Serializers, Partitioners, and Record Accumulators
8 min+15 XP
4.2Producer Batching, Linger, and Compression Tuning (Snappy, LZ4, Zstandard)
8 min+15 XP
4.3Partitioning Strategies: Murmur2 Hashing, Uniform Sticky Partitioner, and Key-Skew Mitigation⚔️ Landmark Capstone
9 min+50 XP
4.4Producer Durability & In-Sync Replicas (acks, min.insync.replicas, Idempotence)
8 min+15 XP
5.1Kafka Consumer Groups Explained: How Multiple Consumers Share Workloads
8 min+15 XP
5.2Offset Management & Commit Strategies (commitSync vs commitAsync)
8 min+15 XP
5.3Consumer Rebalancing: Eager vs Cooperative Sticky Protocol⚔️ Landmark Capstone
9 min+50 XP
5.4Troubleshooting Consumer Rebalances: max.poll.interval.ms and Heartbeats
8 min+15 XP
6.1Why Schemas are Critical in Event-Driven Systems: Beyond Unvalidated JSON
8 min+15 XP
6.2Apache Avro vs Protocol Buffers for Kafka: Binary Serialization Deep Dive
8 min+15 XP
6.3Confluent Schema Registry Architecture: Magic Bytes, Wire Format, and Caching⚔️ Landmark Capstone
9 min+50 XP
6.4Schema Compatibility Modes: Backward, Forward, and Full Evolution Rules
8 min+15 XP
7.1Batch vs Stream Processing: The Lambda Architecture vs The Kappa Architecture
8 min+15 XP
7.2Time Semantics in Stream Processing: Event Time, Ingestion Time, and Processing Time
8 min+15 XP
7.3Watermarks in Apache Flink: Handling Late and Out-of-Order Events
8 min+15 XP
7.4Windowing Operations and Stateful Stream Joins: Tumbling, Sliding, and Session Windows⚔️ Landmark Capstone
9 min+50 XP
8.1What is Change Data Capture (CDC)? Query Polling vs Log-Based Replication
8 min+15 XP
8.2PostgreSQL CDC Internals: Logical Decoding, Replication Slots, and Debezium
9 min+15 XP
8.3Hydrating CQRS Read Models and Search Indexes in Real Time with CDC
8 min+15 XP
8.4Debezium Initial Snapshots and Handling Database Schema Migrations (DDL)
8 min+15 XP
9.1The Kafka Idempotent Producer: Producer IDs, Sequence Numbers, and Deduplication
8 min+15 XP
9.2Kafka Transactions: Atomic Multi-Topic and Multi-Partition Writes⚔️ Landmark Capstone
9 min+50 XP
9.3End-to-End Exactly-Once with Apache Flink: Two-Phase Commit Sink and Checkpoints
9 min+15 XP
9.4Building Idempotent Downstream Sinks: Upserts, Deduplication, and Search Sinks
8 min+15 XP
10.1Critical Kafka Metrics to Monitor in Production: Consumer Lag, ISRs, and JMX
8 min+15 XP
10.2Kafka Capacity Planning and Partition Reassignment: Adding Brokers Safely
8 min+15 XP
10.3Tiered Storage in Kafka and Redpanda: Decoupling Compute from Long-Term Storage
8 min+15 XP
10.4Multi-Cluster Kafka Replication: Active-Passive, Active-Active, and MirrorMaker 2
9 min+15 XP