Optimizing Message Queue Consumers, When to Use Caching versus Direct Database Access?
Sophea Lim
·10 views
Hey everyone. So, I'm building a system. We've got these really busy message queues that feed consumers, and those consumers often need to process data and update database records. The big question is when to add a cache between the consumer and the database. And then, what kind of cache. Like, local or distributed? For instance, if a consumer needs to add info to messages using static configuration data that's accessed all the time, a local cache seems like a no-brainer. But what about data that changes more often, stuff like user profiles or product details? We're seeing some lag when consumers hit the database directly for every single message. Would a distributed cache be too much if the data isn't shared by different services, and its just about taking some load off the database for one busy consumer? Are there situations where going straight to the database, even with good indexing, is actually better than caching because of all the trouble with cache invalidation or other things? I'd love to hear what's worked for people.
1 comment