Scalable Agentic System for Antimicrobial Resistance Surveillance on AWS

The article presents a scalable reference architecture designed to simplify antimicrobial resistance (AMR) surveillance by automating the analysis of public sequencing data. It addresses the challenge of turning raw sequencing runs into actionable insights by removing the need for specialized bioinformatics skills and complex SQL queries. The architecture demonstrates a data pipeline that moves information from raw data ingestion through advanced genomics processing and structured storage to a conversational AI query layer, enabling non-specialist researchers to gain insights using plain language.

Core Architectural Stages

The solution is composed of five loosely coupled stages, ensuring data flows unidirectionally and promoting modularity and scalability. These stages handle everything from data acquisition to interactive querying, providing a comprehensive solution for large-scale genomic data processing.

1. Ingestion: Uses Amazon EventBridge Scheduler or REST calls to trigger AWS Step Functions workflows. Amazon ECS on AWS Fargate fetches and converts sequencing runs from the NCBI SRA mirror on AWS. Amazon DynamoDB ensures idempotency by tracking processed isolates to prevent redundant work.
2. Genomics Processing: AWS HealthOmics manages Nextflow workflows for computationally intensive tasks like read quality control (fastp), de novo assembly (SKESA), and resistance gene detection (AMRFinderPlus). This stage is critical for transforming raw data into meaningful genomic information.
3. Harmonization and Storage: An AWS Lambda function normalizes detection output to the hAMRonization schema. AWS Glue jobs then merge this data into idempotent Iceberg tables on Amazon S3 Tables, ensuring data consistency and efficient updates.
4. Analytical Store: Amazon Athena queries the Iceberg tables via the AWS Glue Data Catalog, providing a fast SQL interface for the conversational agent.
5. Conversational Agent and Web Application: An Amazon Bedrock AgentCore agent, augmented with Guardrails, provides natural language querying capabilities over the analytical store. The front-end is a React SPA hosted on Amazon S3 and CloudFront, secured with AWS WAF and authenticated via Amazon Cognito, with API Gateway handling REST and WebSocket communication.

Key Design Decisions for Robustness and Efficiency

Several architectural decisions enhance the system's robustness, efficiency, and ease of use:

• Idempotent Processing: DynamoDB tracks processed isolates during ingestion, and AWS Glue uses MERGE operations for Iceberg tables, preventing data duplication and enabling efficient re-processing or updates.
• Serverless and Managed Services: Extensive use of AWS Lambda, Step Functions, Fargate, HealthOmics, and Bedrock AgentCore minimizes operational overhead and allows for automatic scaling.
• Data Harmonization: Normalizing data to the hAMRonization schema ensures interoperability and consistency across different analysis tools and future integrations.
• Security and Access Control: Amazon Cognito for user authentication, AWS WAF for web application security, and isolated HealthOmics tasks without internet access enhance the overall security posture.