Architecting a Mini Derivatives Exchange: A Modular Monolith Approach
This article details the architectural decisions and implementation strategy for building a mini derivatives exchange, focusing on a modular monolith design. It highlights the importance of upfront architectural planning, clear module boundaries, and breaking down complex components like the matching engine into manageable, verifiable steps. The author shares insights on structuring a system for clarity, maintainability, and future scalability while leveraging AI assistance effectively.
Read original on Dev.to #systemdesignBuilding a financial system like a derivatives exchange requires careful architectural consideration to ensure correctness, performance, and maintainability. The author opted for a modular monolith, a pragmatic approach that balances the simplicity of a single deployable unit with the benefits of strong module encapsulation typically found in microservices. This design choice aims to keep operational overhead low while enforcing clear separation of concerns within the codebase.
Upfront Architectural Planning
A critical first step was creating a comprehensive architecture document before writing any code. This document defined key structural aspects:
- Repository split: Separate frontend and backend, with one deployable per backend service.
- Component responsibilities: Clearly defined roles for the exchange API, demo UI, and future gamification.
- Technology stack: FastAPI for the API, PostgreSQL for persistence, and Binance public API for market data. Justifications for current tech choices and future considerations (e.g., why Kafka wasn't used yet).
- Module boundaries: Core, market_data, orders, matching, positions, and integration, each with a defined job and public interface.
Modular Monolith Design Principles
Key Principle
The modular monolith approach emphasizes strict module boundaries within a single service, where modules expose only public interfaces, preventing direct access to internal implementations. This simplifies operations while providing internal organizational benefits similar to microservices.
This architectural style was chosen for two main reasons: simplifying operations and enabling focused development. Each module, like `OrderService` or `OrderBook`, exposes a well-defined public interface. Internal module implementations are hidden, preventing tight coupling and making the codebase easier to understand and evolve. This also allowed for more effective use of AI tools by providing clear, constrained tasks.
Designing the Matching Engine
The matching engine, the core of any exchange, was broken down into verifiable steps:
- Data Structures: Define `Order`, `Fill`, and `OrderBook` with methods for `add`, `insert`, `cancel`, `best bid`, and `best ask`.
- Matching Logic: Implement the process where a new order loads the order book from the database, processes the match, and generates fills.
- Persistence: Store orders and positions in PostgreSQL. The order book is rebuilt from the database for each new order or cancellation, prioritizing simplicity and correctness over ultra-low latency by avoiding long-lived in-memory state.
Future considerations for a production-ready system include adding a distributed rate limiter (e.g., using Redis) and comprehensive unit and integration tests for critical components like the matching engine to ensure robustness and correctness.