Exactly-once delivery in distributed systems: myth or achievable?
Min-Jun Takahashi
·12 views
the holy grail of distributed systems always seems to be 'exactly-once delivery,' but textbooks typically say it's impossible. kafka, however, often advertises it. what does 'exactly-once' actually mean in the context of kafka streams and connect, and where does it break down in a real-world, end-to-end system?
my understanding is that it's achievable within the kafka ecosystem itself for stream processing, but once you interact with external systems (databases, external apis), true exactly-once becomes practically impossible without careful idempotent design on the consumer side. are people largely relying on idempotency as the pragmatic solution for 'effectively' once processing, rather than striving for theoretical exactly-once? keen to hear practical insights on this.
7 comments