AWS Architecture Blog·September 30, 2026Building a HIPAA-Eligible Agentic AI Orchestrator on AWS
This article details MHK's architecture for SmartProminence AI Orchestrator, a multi-tenant, event-driven framework built on AWS for processing medical documents with agentic AI while maintaining HIPAA compliance. It highlights how MHK achieved significant efficiency gains and accelerated AI feature deployment by centralizing orchestration and leveraging a controller-agent pattern with Amazon Bedrock, SQS, and Fargate for scalable, compliant, and configurable AI workflows across diverse healthcare use cases.
Read original on AWS Architecture BlogMHK faced the challenge of processing vast volumes of medical documents for various healthcare workflows, requiring intelligent document processing and decision support, all under strict HIPAA compliance. Building individual AI systems for each use case was unsustainable due to fragmented infrastructure, lengthy development cycles, and high manual effort. Their solution was the SmartProminence AI Orchestrator, a reusable, multi-tenant agentic workflow framework on AWS.
Architectural Overview: Controller-Agent Pattern and Event-Driven Design
The SmartProminence AI Orchestrator is built around a stateless, event-driven controller-agent pattern. The Agent Orchestration Core, running on AWS Fargate with Spring Boot, serves as the central nervous system, handling job submission, workflow management, LLM proxying, and token management via a REST API. This core is the sole component with direct database access, enforcing strict data access boundaries.
- Workflow Engine Controller: Determines the sequence and dependencies of steps, resolving them into a Directed Acyclic Graph (DAG) using Kahn's algorithm. It pre-creates step executions and dispatches agents layer by layer, polling for completion before advancing.
- LLM Processing Agents: Execute individual steps, performing tasks like input binding, optional vision processing, prompt assembly with context, LLM invocation via Amazon Bedrock, and post-processing for extraction and type coercion. Agents use Java virtual threads for parallel processing within a single step.
Event-Driven Communication with Amazon SQS
All communication between the orchestration core, controllers, and agents is asynchronous and event-driven, utilizing Amazon Simple Queue Service (SQS). The orchestration core places messages on `ControllerInvokeQueue` to trigger workflows and `AgentInvokeQueue` to dispatch individual steps. Each message is secured with a capability token. This design provides several benefits:
- Stateless Processing: Any available instance can pick up pending messages.
- Independent Scaling: Agents scale horizontally via ECS Fargate, decoupled from workflow logic.
- Fault Isolation: A failed task does not block parallel steps; failures are captured in dead letter queues.
- Decoupled Deployment: New agent versions can be deployed without system-wide restarts.
Key System Design Principle
The system's heavy reliance on asynchronous, event-driven communication (SQS) coupled with stateless compute (Fargate) is a foundational pattern for building highly scalable, fault-tolerant distributed systems. This approach minimizes tight coupling, enhances resilience, and allows for independent scaling of different components, crucial for handling variable loads and ensuring high availability.