Building a Serverless, Event-Driven AI Assistant on AWS

The Challenge: Balancing Responsiveness, Context, and Compliance

Pelago, a digital health company, needed an AI assistant to support their care team with personalized member interactions. Key challenges included maintaining long-term conversation context, ensuring human-in-the-loop oversight, adhering to Protected Health Information (PHI) requirements by keeping data within their AWS VPC, and providing instant suggestions to coaches despite potentially long Large Language Model (LLM) processing times. The solution also needed to be rapidly developed and scalable.

Event-Driven Serverless Architecture for Decoupling and Scale

The core of Pelago's solution is an event-driven serverless architecture utilizing AWS services. This approach decouples concerns and allows for asynchronous processing of computationally intensive tasks, such as AI inference. By treating each incoming member message as an event, the system can fan out processing to independent consumers without blocking the user experience.

1. Asynchronous Processing: Member messages are published to an Amazon SNS topic, which then fans out to multiple AWS Lambda subscribers. This prevents LLM inference (which can take several seconds) from blocking the user interface.
2. Decoupled Services: Each subscriber Lambda handles a specific task (e.g., metadata storage, analytics, AI suggestion generation), allowing independent development, deployment, and scaling without affecting other components.
3. Organic Scaling: AWS Lambda automatically scales horizontally based on traffic, eliminating the need for manual provisioning or scaling configuration, crucial for variable healthcare workloads.
4. Data Security & Compliance (PHI): All AI integrations operate within the existing Amazon VPC infrastructure, ensuring PHI never leaves Pelago's AWS environment and is not exposed to the public internet. Each member's PHI is processed separately.

Key Components and Workflow

The architecture involves several AWS services working in concert:

• AWS AppSync: Receives member messages.
• AWS Lambda: Functions handle message storage, publishing, and subscription-based processing.
• Amazon DynamoDB: Stores full conversation history for quick retrieval (single-digit millisecond performance).
• Amazon SNS: Acts as the message bus for event fan-out, delivering messages to multiple subscribers.
• Amazon Bedrock: Invoked by the Chat Assistant Lambda for AI-powered contextual suggestion generation.
• Amazon RDS (MySQL): Stores pre-generated AI suggestions for rapid retrieval.
• Amazon API Gateway: Serves pre-generated suggestions to the care team front end.

When a coach opens a conversation, pre-generated suggestions are retrieved from MySQL via API Gateway and Lambda in under 100 milliseconds, ensuring a responsive user experience regardless of the LLM generation time. This read-your-writes consistency is achieved by storing the results of asynchronous processing for synchronous reads.