Menu
Back to Discussions

Real-time Analytics: When Do We Need Dedicated Infrastructure?

Liya Tan
Liya Tan
·19835 views
Hey folks, I've been thinking about real-time analytics a lot lately. We're processing a good chunk of event data right now, but our current batch processing setup is getting pretty old for some dashboards. I'm wondering when it's really worth it to invest in dedicated real-time streaming stuff, you know, like Kafka, Flink, or Kinesis/Spark Streaming. Should we stick with optimizing our current batch jobs, or maybe just do some simpler polling? What are the big signs or problems that make you think, "Okay, it's time to go all-in on real-time?" Are there specific amounts of data, latency needs, or particular uses that make it a no-brainer? Really curious to hear what you all have experienced.
40 comments

Comments

Loading comments...
Real-time Analytics: When Do We Need Dedicated Infrastructure? | SysDesAi