Producer Durability & In-Sync Replicas (acks, min.insync.replicas, Idempotence)

In distributed streaming architectures, achieving zero data loss requires precise coordination between producer durability settings (acks), broker replication semantics (min.insync.replicas), and cluster consensus protocols.

A misunderstanding of these configurations is the single most common cause of silent message loss and corrupted data streams in production Kafka clusters.

---

1. Producer Acknowledgement Modes: acks=0 vs acks=1 vs acks=all

The producer parameter acks dictates how many replica brokers must commit a record to their local write-ahead log before the leader broker returns a successful acknowledgement (ACK) to the producer's Sender thread.

Detailed Breakdown of acks Configurations:

---

2. In-Sync Replicas (ISR) and min.insync.replicas

A partition with a Replication Factor of 3 ($RF=3$) has one Leader and two Follower replicas spread across three distinct failure domains (availability zones or racks).

What Defines an In-Sync Replica?

A follower is considered In-Sync if:

1. It maintains an active TCP session with the cluster metadata quorum (KRaft / ZooKeeper).
2. It fetches records from the leader within the time window configured by replica.lag.time.max.ms (default: $30,000\text{ ms}=30\text{ s}$).

If Follower 2 suffers GC pause or network degradation exceeding 30 seconds, the Leader unilaterally expels Follower 2 from the In-Sync Replicas (ISR) list.

---

3. The Fatal Antipattern: acks=all with min.insync.replicas=1

A dangerous misconception is that setting acks=all on the producer guarantees replication across multiple machines.

[!CAUTION] If a topic has min.insync.replicas=1, and two of your three brokers crash, the ISR shrinks to $\{\text{Leader}\}$. The leader alone satisfies acks=all because all replicas in the current ISR (which is just the leader) confirmed the write! If that lone leader then fails, all recent messages are lost.

The Production Durability Golden Ratio:

For any mission-critical production deployment:

• Replication Factor: $RF=3$
• Producer acks: acks = all (-1)
• Broker / Topic min.insync.replicas: min.insync.replicas = 2

Under this configuration:

• The system tolerates the failure of any 1 broker with zero downtime and zero data loss.
• If 2 brokers fail simultaneously, the lone surviving leader refuses to accept new writes, throwing NotEnoughReplicasException to producers, preserving strict consistency over availability (CP system under CAP Theorem).

---

4. unclean.leader.election.enable: Availability vs Data Integrity

When all ISR replicas crash, leaving only out-of-sync replicas alive:

• unclean.leader.election.enable = false (Default & Strongly Recommended): Prevents stale replicas from ever becoming leader. Producers and consumers receive errors until an in-sync broker comes back online.
• unclean.leader.election.enable = true: Sacrifices data consistency for uptime. Stale replicas become leader, permanently truncating all records written to the old leader that had not replicated to the stale node.

---

5. The Idempotent Producer: Eliminating Duplicates & Preserving Order

Under network failures, producers retry sending batches. In standard non-idempotent mode, retrying an unacknowledged write that actually succeeded on the broker results in duplicate records and broken sequence order.

How Idempotence Works Internally:

1. Producer ID (PID): On startup, the producer is assigned a globally unique 64-bit PID by the cluster coordinator via InitProducerIdRequest.
2. Monotonic Sequence Numbers: Each ProducerBatch sent to a specific TopicPartition is stamped with a zero-indexed sequence number (seq = 0, 1, 2, ...).
3. Broker Deduplication Cache: Brokers track the last 5 sequence numbers per PID in memory for each partition. If a batch arrives with $\text{Seq}\le \text{LastSeenSeq}$, the broker acknowledges it without appending a duplicate record.
4. Out-of-Order Rejection: If a batch arrives with $\text{Seq}>\text{LastSeenSeq}+1$, the broker returns OutOfOrderSequenceException, preserving strict partition ordering even when max.in.flight.requests.per.connection = 5.

---

6. Enterprise Durability Configuration Profile

---

7. Summary Checklist

• acks=all configured on all financial, transactional, and audit event producers.
• min.insync.replicas=2 configured on all topics with $RF\ge 3$.
• enable.idempotence=true enabled to prevent retry duplication and race conditions.
• unclean.leader.election.enable=false enforced to prevent catastrophic log truncation.