Software Architecture and System Design News
Latest curated articles from top engineering blogs
Shopify
AWS2003 articles
Zero-Copy, Multi-Protocol Storage for Data Pipelines
This article introduces the "zero-copy, multi-protocol storage" pattern, which eliminates the need for data synchronization jobs in hybrid cloud architectures. By allowing legacy systems and modern cloud services to access the same data volume via different protocols (e.g., SMB/NFS and S3 API), it reduces latency, cost, and complexity. The pattern is demonstrated with Amazon FSx for NetApp ONTAP to achieve real-time data processing for systems not originally designed for it.
Optimizing LLM Inference with Speculative Decoding
This article explores speculative decoding, a technique to significantly speed up Large Language Model (LLM) inference by converting underutilized GPU capacity into output. It details how a smaller draft model generates candidate tokens, which are then verified in parallel by the larger target model, reducing sequential forward passes and maintaining output quality. This method addresses the memory bandwidth bottleneck in autoregressive decoding.
Scaling Pinterest's Home Feed Conditional Learned Retrieval System
This article details Pinterest's architectural evolution of its Conditional Learned Retrieval (CLR) system for the home feed, focusing on how they scaled its retrieval capabilities across diverse use cases and improved model foundations. It covers critical system design aspects such as efficient serving infrastructure, large-scale embedding management, and the unification of condition types to reduce engineering overhead.
The Overlooked Importance of Performance in System Design
This article, featuring Casey Muratori, argues that software performance is often critically overlooked in the industry despite its direct impact on business bottom lines and user experience. It advocates for integrating performance considerations early into system design and architecture, moving beyond reactive profiling, and fostering a deeper understanding of hardware capabilities for truly performant systems.
IBM's Dual-Architecture Processor: Unifying IBM Z and Arm Ecosystems for Enterprise Computing
IBM's new dual-architecture processor aims to bridge the gap between traditional IBM Z mainframes and the growing Arm ecosystem. This innovation allows enterprises to run Arm-native Linux applications alongside z/OS and Linux on IBM Z, leveraging the security, reliability, and scalability of IBM's enterprise platforms while expanding software choice and accelerating AI workloads.
Specification-Driven Composition for Flexible Data Workflows on AWS
This article introduces a specification-driven composition pattern for building highly flexible and maintainable data transformation pipelines, particularly relevant for complex and evolving data environments. It decouples workflow intent from processing logic, enhancing reusability, traceability, and governance within data architectures. The AWS implementation leverages serverless technologies like Lambda and Step Functions to dynamically assemble and execute data workflows based on declarative specifications.
Designing a Scalable Distributed Cache with Consistent Hashing and L1 Caching
This article details the architectural evolution from a monolithic Redis cache to a resilient distributed caching system. It emphasizes solving the "thundering herd" problem and hot-spotting through the adoption of consistent hashing for sharding and incorporating a local L1 cache within service instances. The design aims to minimize network latency and improve scalability and fault tolerance.
Optimizing AI Usage for Cost Efficiency in Large-Scale Systems
This article details Datadog's strategies for managing AI inference costs, crucial for large-scale systems integrating AI. It focuses on agent evaluations, spending guardrails, and context optimization to balance performance and expenditure, offering insights into architectural decisions for sustainable AI deployment.
AWS Architecture Blog·5h agoGallup AI: Scaling Real-time Coaching with a Serverless RAG Architecture on AWS Bedrock
Gallup built an AI assistant, Gallup AI, to provide personalized leadership coaching by leveraging 90 years of proprietary workplace research. This system design focuses on a serverless architecture utilizing Amazon Bedrock for generative AI, RAG through Knowledge Bases and Kendra, and various AWS services for scalability, real-time responses, and data management. It demonstrates how to integrate large language models (LLMs) with private data for domain-specific AI applications.
Ramp's Internal AI Coding Agent: Architecture and Design of 'Inspect'
This article details Ramp's decision and architectural choices behind building 'Inspect', an in-house AI coding agent. It explores why a custom solution was preferred over third-party tools, focusing on the system's ability to provide remote sandboxed development environments with deep internal integrations for verification and autonomous debugging. The design emphasizes extensibility, allowing engineers to build hundreds of specialized agents on top of the Inspect platform.
Continuous Delivery Challenges for Foundational Platforms
This article discusses why traditional Continuous Integration/Continuous Delivery (CI/CD) practices often fail for foundational, stateful platforms compared to typical application development. It highlights the unique challenges faced by platform teams, such as managing state, direct interaction with infrastructure, and higher impact of outages, which necessitate different deployment strategies for ensuring reliability and mitigating blast radius.
Architecting Golden Paths for AI Agents in System Design
This article explores the architectural considerations for creating "Golden Paths" for AI agents, focusing on how system design principles must adapt when the primary users are autonomous agents rather than humans. It emphasizes the need for well-defined execution patterns, machine-consumable contracts, and robust dispatch controls to ensure agents operate reliably and efficiently within a larger system.