API Gateway Aggregation for Microservices, When Does it Become an Anti-Pattern?

So, we're busting up this big old app into smaller pieces, microservices you know. And a lot of the time, it looks like an API Gateway is the way to go for pulling data from a bunch of different services. It seems pretty neat at first, makes things less complicated for the client and cuts down on network chatter. Like, imagine one 'GetUserDetails' spot that grabs info from the 'User Profile', 'Order History', and 'Shipping Address' services, then mashes it all up before sending it back. But I'm starting to worry, where do we draw the line? Are we just making the gateway do all the hard work instead of the client? How can you tell when an API Gateway is getting too chunky, turning into a 'mini-monolith' itself? What tricks do you use to stop that from happening, or when do you just say, 'hey, client, you handle it' and let them talk to services directly?