Mapfre USA's Fraud Detection Platform on AWS: A Lakehouse and ML Approach

Architecting for Complex Fraud Detection

Traditional fraud detection, often reliant on rules-based systems and structured data, struggles with sophisticated fraud rings that involve hidden relationships across entities like policyholders, vehicles, and providers. Mapfre USA addressed this by building a modern data platform, "Atenea," on AWS. This platform combines traditional structured data analysis with graph-based features, processed by machine learning models to identify intricate, non-obvious fraud patterns.

Core Components of the Atenea Data Platform

The Atenea platform is a robust data lakehouse architecture designed for scalability, governance, and efficient data processing. It leverages key AWS services and open-source technologies to handle diverse data types and complex ML workflows.

• Data Lakehouse Foundation: Built on Apache Iceberg tables stored on Amazon S3, with metadata managed by AWS Glue Data Catalog and access governed by AWS Lake Formation. This provides a flexible, scalable, and schema-evolvable data storage layer.
• Layered Data Architecture: The platform employs a three-tier logical data architecture: Silver (raw, standardized source data), Gold (intermediate, unified data like Guidewire logs and features), and Platinum (feature store-managed Iceberg tables for encoded features and model predictions). This layered approach ensures data quality, reusability, and strong metadata governance.
• Elastic Compute: Processing pipelines run on Amazon EMR Serverless for Apache Spark workloads, providing elastic and cost-efficient compute for both batch processing and fast-time scoring. Orchestration is handled by Apache Airflow on Amazon Managed Workflows for Apache Airflow (Amazon MWAA).
• Graph Enrichment: Neo4j is integrated for graph database capabilities, enabling the extraction of advanced network-based features (e.g., suspicious claim linkages, provider fraud ratios, centrality metrics) that are crucial for detecting complex fraud rings.

Resilient MLOps and Guidewire Integration

A critical aspect of the solution is the seamless and resilient integration of ML predictions with the Guidewire Claims system, ensuring that fraud alerts are actionable for front-line adjusters. This closed-loop system is essential for realizing business impact.

1. ML model scoring results are written as JSON files to an S3 path.
2. An S3 event notification triggers an AWS Lambda function.
3. The Lambda function reads the JSON, calls the Guidewire Predictive Model API for each individual payload (due to Guidewire's lack of batch support), and manages retries.
4. Failed requests are sent to an Amazon SQS Dead-Letter Queue (DLQ), and Amazon SNS notifications are published for monitoring and alerts.
5. AWS Secrets Manager is used for secure storage and injection of credentials and API endpoints, ensuring environment-specific and Region-specific access controls.

Monitoring, Data Quality, and Security

The platform emphasizes data quality, resilience, and security. Data quality checks are applied throughout ingestion pipelines and graph feature generation to detect anomalies early. Monitoring dashboards track KPIs and model performance using Amazon CloudWatch and Amazon SNS for alerts. AWS Secrets Manager provides secure management of credentials and tokens for Guidewire API integration, with strict environment and region-specific access controls. This comprehensive approach ensures reliable, transparent, and secure production execution.