Picking a Data Store for Real-time Analytics: OLAP versus Time-Series versus Key-Value?
Solveig Andersen
·14 views
Hey everyone, I'm trying to figure out the best place to store data for a new real-time analytics platform. We're getting a ton of events, like millions every second. And we need to query that data quickly for dashboards, pulling it together over different time periods and categories. I've been checking out a few ideas: regular OLAP databases, databases just for time-series data, and even some clever ways to use a key-value store with good indexing. Each one seems to have good and bad points for how fast you can write, how flexible your queries can be, and how much work it is to keep running. So, has anyone built something like this that handles a lot of data? I'd love to hear what you think works best, or what problems you ran into with whatever you picked. Specifically, when would you go with an OLAP option like ClickHouse instead of a time-series database like InfluxDB or TimescaleDB? And are there times a fast key-value store, maybe RocksDB or ScyllaDB, would actually be better for this kind of work?
3 comments