Proactive Autoscaling for Latency-Sensitive Edge Applications in Kubernetes

Kubernetes HPA, while effective in cloud environments, proves insufficient for the unique demands of edge computing. Edge applications require extremely low-latency, high elasticity, and predictable performance under large, unpredictable spikes in workload. Resource constraints at the edge make efficient scaling critical, but HPA's reactive, formulaic approach often leads to over-scaling, under-scaling, or replica oscillation, impacting performance and wasting valuable resources.

Limitations of Kubernetes HPA at the Edge

• Lack of algorithm flexibility: HPA's hard-coded proportional scaling formula prevents custom logic, time-aware scaling, gradual scale-down, or rate-limiting scale-up, which are crucial for bursty edge workloads.
• Inefficient handling of short-lived spikes: HPA treats all bursts as sustained load, leading to rapid, excessive scale-up and subsequent resource waste.
• Operational overhead of custom metrics: While HPA supports custom metrics, it requires deploying additional components (metrics server, Prometheus, adapters), adding complexity and resource consumption, which is challenging in constrained edge environments.

Designing a Custom Pod Autoscaler (CPA) for Edge

To overcome HPA's inflexibility, a Custom Pod Autoscaler (CPA) is proposed, designed to be context-aware and proactive. The CPA's evaluation algorithm leverages best practices from cloud service providers and SRE teams, moving beyond rigid numeric thresholds to use a composite of three primary workload condition signals:

1. CPU Headroom: Maintains a target utilization safety zone (e.g., 70-80%) to absorb unpredictable bursts without latency impact. It proactively scales up if average CPU usage consistently exceeds this threshold to restore the buffer.
2. Latency SLO Awareness: Integrates p95 or p99 response time as an early indicator of overload. If latency approaches or exceeds defined Service Level Objectives (SLOs), the autoscaler increases replicas proportionally, addressing scenarios where CPU alone is not sufficient (e.g., I/O-intensive workloads).
3. Pod Startup Compensation: Accounts for longer container startup times typical at the edge due to lower disk throughput or image coldness. The autoscaler triggers proactive scaling based on estimated startup times and anticipated load increases to ensure capacity is available before demand outstrips it.