LLM-Native Recommendation System Architecture at Netflix: GenRec
This article details GenRec, Netflix's LLM-backed recommendation ranker, which shifts from traditional hand-crafted features to an LLM-native approach. It highlights the architectural decisions and trade-offs made to integrate Large Language Models into a high-scale production recommendation system, focusing on efficiency, personalization, and alignment with business goals. The system's two-phase training framework, context engineering, and cost-optimization strategies for serving LLMs are key aspects discussed.
Read original on Netflix Tech BlogIntroduction to LLM-Native Recommendations
Netflix's GenRec system represents a significant architectural shift in recommendation engines. Traditionally, these systems rely on extensive feature engineering and specialized models. GenRec leverages Large Language Models (LLMs) to directly process user history and item metadata as text, aiming to reduce complexity and improve adaptability for new content types and product surfaces. This approach capitalizes on LLMs' broad world knowledge and language understanding to capture rich semantic relationships.
GenRec System Architecture Overview
GenRec operates as an LLM-backed recommendation ranker, post-training an internal foundation LLM on Netflix-specific data and objectives. The high-level pipeline involves verbalizing user histories, item metadata, and context into natural-language prompts, which are then fed into the GenRec LLM. This LLM, running on vLLM in prefill-only mode, outputs scores for catalog items to produce a personalized ranking. This design minimizes reliance on hand-engineered features and emphasizes 'context engineering'.
Key Architectural Principles of GenRec
GenRec's architecture is built around several core principles: verbalizing data as text, adapting a foundation LLM for ranking, adding a catalog-aware scoring head, utilizing reward signals for alignment, and optimizing for cost efficiency through prefill-only inference on their LLM serving stack.
Two-Phase Training Framework
- Phase 1 — Netflix-Adapted Foundation LLM: An open-source LLM is adapted on proprietary Netflix corpora to learn content understanding, member behavior patterns, and general language understanding. This phase is updated infrequently and serves as a shared backbone.
- Phase 2 — GenRec: The foundation model is post-trained on ranking-specific data and objectives to become a high-quality ranking model. This phase focuses on ranking quality, incorporates multiple reward signals, is refreshed more frequently, and is optimized under serving cost constraints.
Context Engineering and Verbalization
Instead of dense features, GenRec verbalizes rich user histories and context into natural language. Due to LLM token budget constraints at Netflix's scale, context engineering is crucial. This involves selectively retaining high-signal engagements, omitting low-signal events, summarizing repetitive behaviors, and elaborating on important items, all while prioritizing recent, high-signal history within a fixed token budget. This balances information preservation with cost efficiency for prompt processing.
Multi-Objective Training and Alignment
GenRec uses a multi-objective loss combining a recommendation ranking objective, language modeling (LM) objectives, and alignment via reward-weighted training. The catalog-aware ranking objective teaches item scoring, while the LM objective preserves general language understanding. Crucially, a reward-weighted loss aligns the model with long-term member utility and business requirements (e.g., content type balancing), providing a cost-effective alternative to full reinforcement learning.
Serving and Cost Optimization
Serving LLMs at Netflix scale is cost-intensive. GenRec addresses this through three primary strategies: 1) using smaller/distilled foundation models; 2) aggressive context compaction via context engineering to minimize tokens; and 3) prefill-only inference. Instead of expensive autoregressive decoding, the model consumes the prompt once and scores the entire candidate set in a single forward pass, making high-volume workloads feasible within compute budgets.