Scaling ML Workloads with Hybrid AWS Architecture for Geological Analysis

Introduction to LITHOLENS™ and the Challenge

LITHOLENS™ by ALS GeoAnalytics automates geological core logging using machine learning and computer vision. This platform addresses significant challenges in traditional mining analysis, such as subjective interpretations by human geologists, remote site access difficulties, underutilized historical data, and scheduling bottlenecks. The goal was to achieve higher accuracy, consistency, and scalability in geological insights.

Machine Learning Pipeline for Geological Analysis

The core of LITHOLENS™ is a robust ML pipeline. It begins with a Color Extraction module identifying unique pixel colors in core images, stored in Amazon S3. This feeds into a Color Clustering module, which uses algorithms like K-Means or Gaussian Mixture Models to reduce image complexity and highlight mineralogical variations. A Percentage Report module then quantifies color composition along the core, enabling spatial analysis. Additionally, the system employs specialized deep learning models like RoQE Net for Rock Quality Designation (RQD) and VeinNet/CobbleNet for identifying complex geological features, demonstrating superior accuracy and scalability over traditional methods.

Hybrid AWS Solution Architecture

ALS GeoAnalytics deployed LITHOLENS™ using a hybrid AWS architecture designed for both performance and cost efficiency. It combines containerized workloads on Amazon EKS for compute-intensive ML tasks with serverless components via AWS Lambda for lightweight API operations. Amazon S3 serves as the primary data lake for input, intermediate, and output data, while Amazon RDS manages structured metadata.

Key Architectural Decisions for Scale and Efficiency

• Amazon EKS for ML Workloads: Chosen for managing deep learning model training and inference, leveraging GPU-accelerated G6 instances with automatic scaling based on job queue depth to handle sustained compute needs.
• AWS Lambda for API Gateway: Used for job submission, status checking, and result retrieval. This serverless approach eliminates the overhead of always-on servers, reducing costs during low-usage periods by only consuming resources during active requests.
• Pre-configured AMIs: Custom Amazon Machine Images (AMIs) are used, containing all necessary dependencies and model artifacts. This significantly reduces container startup times (from minutes to under 30 seconds), improving job throughput and minimizing idle compute costs.
• Automated Resource Management: EKS clusters are configured to scale down to zero when no jobs are queued, ensuring compute resources are only utilized during active processing. This, combined with S3 for data persistence and RDS for metadata, creates a highly cost-effective and scalable architecture.