Software Architecture and System Design News
Latest curated articles from top engineering blogs
Shopify
AWS1820 articles
Netflix GenRec: LLM-Powered Recommendation System Architecture
Netflix's GenRec is a novel recommendation system that leverages Large Language Models (LLMs) to enhance content personalization. It addresses the complexity and cost of traditional feature engineering by using LLMs to interpret user behavior and content in natural language. The architecture involves a two-phase training process to adapt a foundation LLM to Netflix's specific domain and then fine-tune it for ranking recommendations, optimizing for user satisfaction and operational efficiency.
Cross-Cluster Querying for Distributed Log Analysis in BYOC Environments
This article introduces Datadog's cross-cluster query capability for Bring Your Own Cloud (BYOC) logs, enabling unified analysis of log data distributed across multiple cloud regions or business units without centralizing storage. This approach addresses challenges in incident investigation and compliance for geographically dispersed or segmented systems, emphasizing a federated query model over a centralized data lake.
Scaling Systems to Millions of Users: A System Design Interview Approach
This article outlines a structured approach to answering a common system design interview question: scaling a system to millions of users. It emphasizes starting with clear requirements, estimating scale, designing core components, and incrementally adding advanced features and optimizations, making it a highly relevant resource for understanding system design interview methodologies and common scaling patterns.
Pinterest's User Journey Inference with LLMs: A Generative Approach to Personalization
This article details how Pinterest evolved its user journey inference system from a multi-stage clustering pipeline to a single, fine-tuned 4-billion-parameter LLM. The new generative approach reads chronological activity logs to directly output a ranked list of user journeys, significantly improving personalization quality and addressing limitations of the legacy system. The architecture relies on distilling large frontier models into smaller, production-ready student models and optimizing the serving stack for high throughput at notification scale.
Designing Reliable Job Retries with Queues vs. Database Polling
This article explores architectural decisions for handling failed jobs, specifically reservation expiry in a healthtech context. It compares using a message queue with a Dead-Letter Queue (DLQ) and redrive mechanisms against a manual database polling approach. The core trade-off revolves around latency requirements, operational costs, and the complexity of ensuring idempotency, backoff, and poison-message handling.
Netflix's Ontology-Driven Observability: Building an E2E Knowledge Graph for AIOps
Netflix addresses the complexities of observability at scale (38M events/sec) by moving from reactive monitoring to an AI-driven operational ontology. They achieve this by unifying diverse telemetry data (MELT: Metrics, Events, Logs, Traces) into a queryable knowledge graph, enabling automated triaging, root-cause analysis, and proactive issue prediction and resolution. This system design aims to provide seamless end-to-end visibility and reduce incident resolution time by connecting siloed data sources.
System Design for Physical AI (AIoT) Architectures
This article explores system design considerations for Physical AI, or AIoT, which differs significantly from traditional web application architectures. It introduces a "Four-Engine Architecture" designed to address edge constraints like latency and packet loss by decoupling concerns for local processing and asynchronous cloud synchronization. The core challenge lies in building robust edge infrastructure to support local inference and hardware actuation while efficiently managing data flow to the cloud.
Mastering Software Boundaries in System Design
This article emphasizes the critical skill of defining effective software boundaries in system design. It explains that well-defined boundaries encapsulate responsibilities, rules, and dependencies, enabling clear communication pathways between system components. The core idea is to group related concerns and minimize coupling, leading to more maintainable and scalable architectures.
Netflix's Ontology-Driven Observability: Building an E2E Knowledge Graph for AIOps
This article details Netflix's approach to building an end-to-end (E2E) observability system, tackling the challenges of correlating user experience with system performance at massive scale (38M events/sec). They discuss transitioning from reactive monitoring to an AI-driven operational ontology, leveraging knowledge graphs and agentic workflows for automated triaging, root-cause analysis, and self-healing systems. The core challenge is framed as a data engineering problem to unify disparate telemetry data into an actionable knowledge graph.
Autonomous Coding Agents in Software Delivery Platforms
This article discusses Harness's acquisition of Augment Code, integrating autonomous coding agents into their software delivery platform. It highlights how these agents aim to automate code generation, testing, and even issue remediation by leveraging deployment history and feedback from downstream systems. The core system design challenge lies in orchestrating these AI agents, managing context across various stages of the SDLC, and ensuring secure, controlled automation.
Architecting Multi-Agent AI Systems at Scale: Spotify's Advertising Platform
This article discusses Spotify's approach to building a production-grade multi-agent AI platform for advertising, emphasizing architectural patterns for scalability and reliability. It covers key considerations such as agent boundary definition, tool design for LLMs, deterministic guardrails, and tracing-based evaluation strategies. The insights offer valuable lessons for designing and operating complex AI-powered distributed systems.
Database Workload Capture and Replay System for Scalability and Compatibility
Airbnb developed a robust system to capture and replay real production database workloads. This system addresses critical challenges like capacity planning, de-risking database upgrades, and performance debugging by simulating production traffic in test environments. By leveraging ProxySQL as a client-agnostic capture point, the architecture ensures accurate transaction replay and result comparison across different database versions or configurations.