Software Architecture and System Design News
Latest curated articles from top engineering blogs
Shopify
AWS16 articles
MongoDB Innovations for AI-Driven Systems and Cloud Independence
MongoDB's 2025 review highlights their strategic pivot towards AI, with acquisitions like Voyage AI and the launch of MongoDB AMP, focusing on enhancing AI application accuracy and modernizing legacy systems. Key advancements include integrating search and vector search into Community and Enterprise editions, enabling hybrid AI-native application deployments. The article also emphasizes evolving enterprise requirements for high availability, tunable consistency, and cloud independence in data platforms.
RabbitMQ vs. Kafka vs. Pulsar: Choosing the Right Messaging System
This article compares three prominent messaging systems: RabbitMQ, Kafka, and Pulsar, highlighting their distinct architectural models and use cases in distributed systems. It emphasizes that the choice depends on how data should flow, its retention requirements, and consumption patterns, rather than just speed or popularity.
Designing Reliable Data Pipelines: Architecture and Failure Handling
This article outlines a robust architectural approach for building reliable data pipelines, emphasizing that reliability is a design property, not an afterthought. It introduces a four-layer architecture (Ingestion, Staging, Transformation, Serving) and discusses essential design principles like resumability, idempotency, and observability. Key failure handling patterns and dependency management strategies are also presented to ensure data integrity and operational stability.
Pantone's Agentic AI Architecture with Azure Cosmos DB
This article details Pantone's architectural approach to building an agentic AI-powered Palette Generator using Azure. It highlights the critical role of Azure Cosmos DB as a real-time data layer for managing conversational context, user interactions, and prompt data, emphasizing its scalability and flexibility for AI-driven applications. The architecture incorporates a multi-agent system and is designed to evolve towards vector-based workflows for enhanced semantic understanding.
Designing Shared File Storage with Azure Files for Geo-Distributed Offices
This article outlines the architecture and deployment of a highly available and secure shared file storage solution using Azure Files for geographically dispersed corporate offices. It emphasizes balancing performance with security, leveraging Azure's Zone-Redundant Storage (ZRS) for resilience, snapshots for data integrity, and Virtual Networks for zero-trust access control.
Vision RAG: Multimodal Retrieval for Enhanced LLM Context
This article introduces Vision RAG, an evolution of traditional RAG systems designed to enable search and retrieval on complex, multimodal documents beyond plain text. It leverages next-generation multimodal embedding models, like Voyage AI's voyage-multimodal-3, to index visual and textual content simultaneously, overcoming limitations of OCR-based methods for enterprise data. The system design focuses on unified embeddings for efficient vector search and feeding relevant visual assets to vision-capable LLMs for grounded answers.
Scaling Uber: From Monolith to Microservices and Global Scale
This article details Uber's architectural evolution from a LAMP stack monolith to a globally scaled microservices platform, addressing challenges in real-time data processing, concurrency, and data storage. It highlights key technological decisions and custom-built tools for managing complexity inherent in massive distributed systems.
Choosing ORMs for Performance and Developer Velocity in .NET Systems
This article discusses the architectural considerations for choosing between ORMs like EF Core and micro-ORMs like Dapper in modern .NET applications. It argues that the performance gap has narrowed, making developer velocity and architectural clarity more critical factors than raw ORM speed in most production systems. The author highlights that network latency and I/O often dominate request costs, diminishing the impact of micro-optimizations within the ORM layer.
Scaling IP Geolocation with Redis Sorted Sets and Pipelining
This article details the architectural decisions and implementation strategies behind a high-scale IP geolocation service. It focuses on leveraging Redis's partitioned sorted sets and pipelining capabilities to achieve sub-millisecond enrichment for millions of events, addressing challenges like data freshness, query performance, and operational efficiency.
MongoDB Enhancements for Production AI Workloads: Embedding, Vector Search, and Intelligent Assistance
This article from MongoDB.local San Francisco 2026 highlights new capabilities designed to accelerate AI application development and deployment. Key announcements include the Voyage 4 embedding model family, Automated Embedding for MongoDB Community Edition, Lexical Prefilters for Vector Search, and an intelligent assistant in MongoDB Compass. These features aim to simplify data management for AI, enhance retrieval accuracy, and streamline development workflows by integrating AI functionalities directly into the MongoDB data platform.
Kafka's Core Architecture and Design Principles
This article provides an in-depth look into Apache Kafka's fundamental architecture, focusing on its distributed log design, performance optimizations for high throughput and data persistence on HDDs, and its evolution from ZooKeeper to KRaft for distributed consensus. It highlights Kafka's role as a central nervous system for real-time data streaming in large-scale distributed systems.
Optimizing Cloud Database Costs with Centralized Resource Management
This article discusses how Cambia Health Solutions leveraged Datadog Cloud Cost Management and Resource Catalog to achieve significant savings by optimizing their Amazon RDS instances. It highlights strategies for standardizing database configurations and improving Reserved Instance utilization across a large cloud environment, offering insights into operational cost management within cloud architecture.