Database vs. Message Queue for Audit Logs, A Trade-off Discussion
Solveig Andersen
·1465 views
Okay, so you're building a new service and need to log a ton of user actions. I mean, millions of events every day. It's crucial these logs don't disappear, and the order matters for each user. You don't need to read them right away, though. You're thinking about just writing them straight to a database, maybe a NoSQL one for some wiggle room. But then you're also wondering if sending them to a message queue first, and having another process save them to a cheaper, more analytics-friendly place, makes more sense. What's the deal with the pros and cons here? Is a direct database write good enough if you set it up right? Or are the benefits of a message queue, like separating things, dealing with too much data, and future possibilities (like real-time analytics or spotting weird stuff), too good to pass up, even if it's a bit more complicated?
40 comments