What's the best way to monitor data pipeline health and ensure data quality?

Hey everyone, I'm working on improving our data platform and I'm looking for input on monitoring data pipelines. We've got a mix of batch and streaming jobs, and sometimes issues pop up that aren't immediately obvious from basic job status checks. I'm curious: what are your preferred strategies for monitoring data pipeline health beyond just 'succeeded' or 'failed'? Specifically, how do you approach data quality checks within your monitoring strategy? Are there particular tools or patterns that have worked well for you to catch things like schema drift, data freshness issues, or unexpected value distributions early on?