Gallup AI: Scaling Real-time Coaching with a Serverless RAG Architecture on AWS Bedrock

The Challenge: Centralizing and Personalizing Decades of Research

Gallup possessed extensive workplace science research but lacked a centralized, real-time mechanism to deliver personalized guidance to leaders. The goal was to transform this knowledge into an instantly accessible AI assistant, integrating it directly into their existing Gallup Access application workflow. This required a system capable of semantic search over a vast knowledge base and generating contextually relevant responses at scale.

Architectural Overview of Gallup AI

The Gallup AI assistant is built on a serverless-first architecture designed for high scalability and low operational overhead. It primarily leverages Amazon Bedrock for its generative AI capabilities, including access to Anthropic's Claude models, and integrates with other AWS services to form a robust RAG (Retrieval Augmented Generation) system. This approach allowed Gallup to move from prototype to production rapidly without a dedicated ML Ops team.

Key Components and Their Roles

• Amazon Bedrock: Provides access to foundation models (like Anthropic's Claude) for text generation, along with managed RAG capabilities via Knowledge Bases and content safety guardrails.
• Amazon Bedrock Knowledge Bases & Amazon Kendra: Used for retrieving relevant proprietary research and indexed website content, respectively. This dual approach ensures grounding in both historical archives and current insights.
• AWS Lambda with FastAPI: Serves as the compute layer, handling both real-time streaming responses for web clients and synchronous requests for backend services.
• Amazon ElastiCache Serverless: Caches recent conversation history, enabling sub-millisecond retrieval for a smooth user experience.
• Amazon RDS for MySQL: Acts as the durable system of record for conversations, prompts, responses, and source citations, ensuring transactional integrity.
• Amazon DynamoDB: Stores product-specific insights and contextual data for personalized user experiences.
• AWS Systems Manager Parameter Store: Centralized configuration management for AI model settings, content safety policies, and performance thresholds, allowing dynamic adjustments.
• Amazon Data Firehose to Amazon S3: Streams comprehensive usage metrics (tokens, latency, etc.) for cost management, performance optimization, and observability.

Data Flow and Processing

When a leader queries Gallup AI, the system orchestrates several steps: proprietary workplace research (stored in S3) is ingested into Amazon Bedrock Knowledge Bases, and the Gallup website content is continuously crawled and indexed by Amazon Kendra. Upon receiving a query, relevant documents are retrieved from both sources, scored, filtered, and consolidated before being passed to the Claude models in Amazon Bedrock for generating a response. AWS Lambda handlers manage this flow, while ElastiCache provides fast access to conversation history, and RDS ensures persistent storage of interactions. Bedrock Guardrails are applied during generation for content safety.