Menu
Back to Discussions

Exactly-once delivery in distributed systems: myth or achievable?

Raj Ivanova
Raj Ivanova
·925 views
The concept of 'exactly-once delivery' in distributed systems is often called a myth in textbooks, yet Kafka claims to achieve it. I'm trying to understand what Kafka actually guarantees and where the boundaries of that guarantee lie. From what I understand, Kafka's exactly-once semantics apply within a Kafka Connect pipeline or stream processing job (like Flink or Spark Streaming) when reading from and writing back to Kafka. However, once messages leave Kafka and interact with external systems (e.g., updating a database or calling a third-party API), true end-to-once delivery becomes incredibly difficult, if not impossible, due to network failures, service outages, and idempotent operations being a prerequisite. It feels like the practical solution often boils down to ensuring idempotency in our downstream consumers rather than relying on a global 'exactly-once' guarantee. What are the practical implications of Kafka's exactly-once feature, and what are the realistic limitations when integrating with external services?
0 comments

Comments

Sign in to join the conversation.

Loading comments...