Production Architecture Patterns for Eclipse Dataspace Components on AWS

Deploying Eclipse Dataspace Components (EDC) in production environments on AWS requires careful architectural considerations to ensure isolation, leverage managed services, and implement layered security. EDCs facilitate secure and sovereign data sharing by acting as connectors within a data space, adhering to standards like those from the International Data Space Association (IDSA). This article provides a deep dive into practical deployment patterns for production use cases on AWS, building upon fundamental concepts of data space architectures.

Fundamental Architecture Building Blocks

An EDC connector typically comprises a control plane and a data plane, deployed as containers. A robust production architecture on AWS leverages various managed services to support these components. The core architecture uses:

• Amazon ECS and AWS Fargate: For serverless container orchestration, providing scalability and reducing infrastructure management overhead.
• AWS Secrets Manager, Amazon Aurora, and Amazon Cognito: For persistence of secrets and relational control plane data, and for managing OAuth 2.0 client credentials as managed services.
• Amazon S3: For durable data storage to handle both inbound and outbound data transfers securely.
• Amazon API Gateway and Network Load Balancer: To provide secure, private network connectivity to EDC APIs within an isolated Amazon VPC using VPC links, exposing selected API resources while protecting access with AWS IAM and SigV4.

Key Principles for Production-Readiness

The recommended architecture adheres to key principles from the AWS Well-Architected Framework, ensuring operational excellence, strong security, and high reliability.

• Operational Excellence:
• * Infrastructure as Code (IaC): Using tools like AWS CDK for consistent, repeatable, and version-controlled deployments. This also aids in "shifting security left" by validating configurations early.
• * Observability: Integrating Amazon CloudWatch Container Insights and Logs, alongside EDC's health check endpoints, for comprehensive visibility and proactive issue detection.
• * Managed Services: Prioritizing AWS managed services (Aurora, Secrets Manager, Fargate, Cognito) to offload operational burden, gain high availability, and leverage built-in security and compliance.
• Security:
• * Defense in Depth: Implementing multiple layers of security, including network isolation (VPC private subnets), security group segmentation, IAM least privilege, and encryption everywhere (at rest, in transit, and during processing).
• * Principle of Least Privilege: Granting only necessary permissions to each component and restricting API exposure through API Gateway and internal Network Load Balancers.
• Reliability:
• * Fail Fast, Recover Automatically: Designing systems to detect failures quickly and recover without manual intervention, utilizing ECS circuit breakers, automated health checks, and point-in-time recovery.
• * Regional Resilience: Distributing components across multiple Availability Zones with cross-zone load balancing and Aurora's multi-AZ replication to tolerate AZ failures.
• * Decoupled Components: Separating control plane and data plane as independent services with distinct responsibilities for independent scaling, updates, and failure isolation.