how do you design a system for tons of unique, short-lived files

im workin on a system that needs to ingest and serve a massive volume of small files, each potentially existing for only a few minutes. think temporary session data, a/b test configs, or real-time analytics events. traditional object storage or even file systems seem like overkill and might struggle w/ the sheer number of objects and churn. any thoughts on architectures or specific techs that handle this type of workload well? what are the tradeoffs between in-memory solutions, key-value stores, or specialized distributed file systems in this scenario