Pinterest's 3-Tower Engagement Co-Train Model for Ads Ranking
This article details Pinterest's architectural evolution for ads lightweight ranking, moving beyond traditional two-tower models to a more sophisticated three-tower co-train approach. It highlights the system design challenges of balancing model complexity with stringent latency and cost constraints at Pinterest's scale, especially when scoring hundreds of thousands of ad candidates per request. The solution involves a hybrid serving flow that leverages both fast, less accurate two-tower predictions and richer, slower three-tower predictions.
Read original on Pinterest EngineeringPinterest's journey to improve ads lightweight ranking showcases a classic system design trade-off: enhancing model accuracy and expressiveness versus maintaining low latency and cost at massive scale. Building on their Nexus serving stack, which decoupled candidate generation from scoring, they introduced a "3-tower co-train model" to predict multiple engagement probabilities (pCTR, pGCTR30, pOCTR) simultaneously. This model design addresses the challenge of scoring hundreds of thousands of candidates per request, where simply increasing model complexity is not feasible.
Hybrid Scoring Architecture
The core innovation lies in a two-stage scoring flow that intelligently combines two-tower and three-tower predictions within a single co-trained model. This allows for:
- Fast Two-Tower Scores: Initial, less accurate scores for all candidates, leveraging cached Pin embeddings and fast dot-product predictions. This acts as a coarse filter or initial ranker.
- Richer Three-Tower Scores: More accurate scores for a selected subset of candidates, enabling more complex cross-attention and interaction modules. This is used for fine-grained ranking where the additional compute yields the greatest impact.
This tiered approach is critical for managing the computational load associated with higher-order interactions while still meeting strict latency requirements for a large candidate pool.
Model Architecture and Key Components
The 3-tower model extends an existing two-tower setup with two main additions:
- Cross Tower: Utilizes reduced-query cross-attention between user sequences and candidate features to capture high-order interactions. Design choices here involved balancing expressive attention patterns (longer sequences, more attributes) with latency constraints. They converged on using candidate-side features to generate query tokens, interacting with user sequences for predictable compute.
- Inter Module: A shared-bottom MLP design that processes query, Pin, and cross embeddings to produce a compact, shared representation for all three engagement tasks (CTR, gCTR30, oCTR). This module allows for capturing shared patterns across tasks while maintaining task-specific flexibility, chosen for its balance of performance gains and modest latency increase over more complex MMoE architectures.
System Design Lesson
When designing large-scale ML serving systems, especially for ranking, consider a multi-stage retrieval and ranking pipeline. Start with simpler, faster models (e.g., two-tower) for initial candidate filtering or coarse ranking, then apply more complex, accurate models (e.g., three-tower, re-rankers) to a smaller, more relevant subset of candidates to balance performance and latency.
Serving Performance Optimizations
To make the complex three-tower model viable for serving, several critical optimizations were implemented to reduce P99 model inference latency from over 200 ms to about 30 ms:
- Offline Pin Pre-processing: Moving cross-attention related feature pre-processing into the Pin tower, caching enriched Pin embeddings offline. This eliminates per-request MLPs and saves significant latency at serving time.
- Lower Precision Inference (BF16): Switching from FP32 to BF16 for the three-tower path significantly reduced inference time without impacting model quality, making it practical for high candidate volumes.