Flink 1.19 Beats Kafka Streams 3.8 and Spark 4.0 in 10‑Second Tumbling Window Benchmark
Benchmark Flink 1.19 shows 1.82 M events/sec for 10‑second tumbling windows, outperforming Kafka Streams 3.8 and Spark 4.0 by 3.2×; consider using Flink for high‑throughput pipelines.
Test your current streaming workloads on Flink 1.19 to evaluate throughput gains; compare with existing Kafka Streams or Spark setups.
Summary
A 2024 survey of 1,200 data engineers found that 68 % of production stream‑processing outages are caused by misconfigured or under‑performing window operations. The article benchmarks Flink 1.19, Spark 4.0, and Kafka Streams 3.8 on identical hardware: three AWS c7g.4xlarge workers (16 vCPU, 32 GB RAM, Graviton3), a single Kafka 3.8 broker, OpenJDK 17.0.9, and a 10 Gbps VPC network. Flink 1.19 delivers 1.82 M events per second for a 10‑second tumbling‑window count aggregation, 3.2× faster than Kafka Streams 3.8 (567 k eps) and 2.1× faster than Spark 4.0 (870 k eps).
Kafka Streams 3.8 will gain native RocksDB 8.x support in 3.9, which is expected to close the throughput gap with Flink by roughly 18 % according to early‑access builds. Spark 4.0 reduces total cost of ownership by 22 % for hybrid batch‑window workloads by reusing existing Spark ML and SQL libraries, even though its pure streaming throughput remains lower. Flink’s state back‑ends include RocksDB, HashMap, and Heap, while Spark supports RocksDB, HDFS, and S3, and Kafka Streams uses RocksDB and in‑memory. Operational overhead scores are 4 for Flink, 3 for Spark, and 1 for Kafka Streams, reflecting the need for separate clusters versus library‑based deployment.
Key changes
- Flink 1.19 achieves 1.82 M events per second for 10‑second tumbling windows
- Kafka Streams 3.8 achieves 567 k events per second, 3.2× slower than Flink
- Spark 4.0 achieves 870 k events per second, 2.1× slower than Flink
- Kafka Streams 3.9 will add native RocksDB 8.x support, closing the throughput gap by ~18 %
- Spark 4.0 reduces total cost of ownership by 22 % for hybrid batch‑window workloads
- Flink supports RocksDB, HashMap, and Heap; Spark supports RocksDB, HDFS, and S3; Kafka Streams supports RocksDB and in‑memory
- Operational overhead: Flink 4, Spark 3, Kafka Streams 1
- 68 % of outages traced to window operations per 2024 survey