Pinterest's Metrics Board: An End-to-End Metrics Platform

At Pinterest, critical business and experimentation decisions rely on reliable and trustworthy metrics. Faced with thousands of metrics created by hundreds of data producers, Pinterest developed Metrics Board to address the challenges of inconsistent metric definitions, slow pipeline creation, and lack of trust in data. This platform aims to provide a unified, scalable, and resilient solution for metric management at a planetary scale.

Addressing Prior Challenges

• Slow Development: Manual creation of similar data pipelines consumed significant engineering time.
• Inconsistent Quality: Lack of standardized monitoring led to undetected data breakages and low trust.
• Redundancy: Poor discoverability of existing metrics resulted in duplicate efforts and conflicting definitions.
• Metadata Drift: Metric metadata was scattered across systems with no single source of truth.

Metrics Board Architecture Pillars

Metrics Board operates on five core pillars, ensuring an end-to-end solution from definition to consumption.

• Inventory: A single source of truth for metric definitions stored as YAML specs in Git, treating metrics as production code with version control and code review. It opted for accepting raw SQL queries rather than enforcing a strict semantic model due to user feedback and data variety.
• Compute: Each metric definition is transformed into a dedicated Airflow DAG, allowing individual tuning of schedules, retries, and resources. It automatically compiles SQL for Spark/Presto and generates data quality checks, with automated operations for upstream data dependencies.
• Quality: Integrates rule-based checks (e.g., day-over-day thresholds) and ML-based anomaly detection (Warden) directly into metric pipelines. Alerts are routed to owning teams, and a comprehensive quality report is maintained.
• Publishing: Automates integration with downstream systems like PinCat (data catalog), Helium (experimentation platform), and Superset (BI layer), based on flags in the metric definition. Metadata publishes at commit time, data publishes after computation.
• Serving & Discovery: Centralizes rich metadata (ownership, description, certification grade) in PinCat for easy discovery. Metrics carry a certification grade (A+, A, B, C, Unknown) to signal trustworthiness, and Superset dashboards are tiered to dictate retention, performance, and documentation requirements.