In traditional request-response web servers and batch SQL jobs, time is trivial: it is simply the reading of the current machine's system clock (System.currentTimeMillis()).
In distributed real-time stream processing, treating the local system clock as ground truth leads to catastrophic financial miscalculations, corrupted metric rollups, and invalid analytical insights.
To compute accurate aggregations over unbounded data streams, stream processing engines (such as Apache Flink, Kafka Streams, and Apache Spark Structured Streaming) differentiate between three distinct notions of time: Event Time, Ingestion Time, and Processing Time.
1. The Three Tiers of Time Semantics
A. Event Time
- Definition: The exact moment the event physically occurred in the real world, captured and stamped into the event payload by the originating client device, browser, or sensor.
- Characteristics: Immutable and deterministic. No matter how many times you reprocess the stream—whether live in real time or replaying from Kafka 6 months later—the event timestamp never changes.
- Essential For: Financial transaction auditing, session windowing, billing pipelines, and mobile telemetry.
B. Ingestion Time
- Definition: The timestamp recorded by the Apache Kafka broker when the leader node commits the record to its local partition log segment (
CreateTimeorLogAppendTime). - Characteristics: Monotonically increasing per partition. Less prone to client clock skew than event time, but cannot account for delays between user action and network transmission.
C. Processing Time
- Definition: The local wall-clock time of the stream processing node running the operator at the exact instant the record is evaluated in RAM.
- Characteristics: Non-deterministic. Running the exact same stream pipeline twice produces completely different window aggregations due to thread scheduling, GC pauses, and consumer lag.
- Only Used For: Low-overhead real-time monitoring where approximate results are acceptable and deterministic replayability is not required.
2. Why Processing Time Fails: The Mobile Subway Problem
Consider a ride-sharing or fintech application where mobile users generate events while traveling through subway tunnels with intermittent network connectivity:
The Architectural Invariant:
If you rely on Processing Time, network partitions and consumer lag shift historical events into future time windows, destroying time-series accuracy. Event Time guarantees identical results regardless of message transit delays or reprocessing speed.
3. Event Time Skew: Watermarks and Time Drift
Because real-world events experience unpredictable network delays, events almost never arrive in strict chronological order.
In the timeline above, Event C occurred at 08:59:50, but arrived after Event B (09:00:01). Stream processing engines handle this temporal disorder using Watermarks, which track progress in Event Time independently of wall-clock time.
4. Multi-Language Implementation: Event Time Extraction
A. Java (Apache Flink DataStream API)
B. Go (Custom Sliding Window Event Time Evaluator)
5. Summary & Decision Matrix
| Dimension | Event Time | Ingestion Time | Processing Time |
|---|---|---|---|
| Deterministic Replay | 100% Deterministic | Mostly Deterministic | Non-Deterministic |
| Handles Mobile Delays | Yes (Out-of-order buffers) | No | No |
| Clock Skew Sensitivity | Susceptible to bad device clocks | Immune to device clocks | Dependent on worker node clocks |
| Computational Overhead | Requires watermark tracking in state | Low overhead | Zero overhead |
| Recommended Use | Financial, Auditing, Analytics | Monotonic ordering proxies | Transient alerting / Log monitoring |