Meta's AV1 Adoption for Real-Time Communication: System Design Challenges and Solutions

Introduction to AV1 for RTC

Meta's motivation to adopt AV1 for real-time communication stems from its superior compression efficiency, offering at least a 20% bitrate reduction compared to H.264/AVC for the same visual quality. This is particularly crucial for users on slower or limited networks, enabling better video quality at lower bandwidths (e.g., maintaining clarity below 100 kbps). AV1 also provides advanced coding tools like palette mode and intra-block copy, significantly improving performance for screen content with text and repetitive patterns.

Core System Design Challenges in RTC Codec Adoption

Integrating a new codec like AV1 into a large-scale RTC system introduces several architectural and operational challenges that are distinct from Video on Demand (VOD) systems:

• Low Latency Requirement: RTC demands end-to-end latency below 300 milliseconds. Techniques that improve video quality (like multi-pass encoding or extensive buffering) can introduce unacceptable delays.
• Dynamic Network Adaptation: RTC systems must adapt to fluctuating network bandwidth and packet loss. Changes in resolution or retransmissions often require new key frames, leading to bitrate spikes and potential video freezes.
• Computational Complexity & Power Efficiency: AV1's advanced compression comes with higher computational demands for both encoding and decoding, leading to increased power consumption (e.g., 14% more than H.264/AVC on a Pixel 8) and memory usage on mobile devices. This directly impacts battery life and app stability.
• Binary Size: Adding new codec libraries significantly increases application binary size, impacting update success rates, startup times, and overall app health, which is a major concern for billions of users.

Architectural Solutions and Trade-offs

• Low-Complexity Encoder Development: To mitigate the computational burden, Meta developed an internal ultra-low-complexity AV1 encoder preset that achieves encoding complexity comparable to H.264/AVC. This allows AV1 deployment on a much broader range of mid- and low-end devices by dynamically adjusting encoder presets based on device capabilities.
• Optimized Decoder Selection: After comparing several open-source decoders, Meta selected dav1d for its superior power efficiency and reliability, crucial for real-time decoding on mobile.
• Binary Size Optimization: Strategies included using a dynamic-download framework (though limited by network failures), direct library optimizations (e.g., removing unused tools like quantization matrix to save 60 kB), and sharing codec libraries across different application features or leveraging built-in platform support.
• ML-based Device Eligibility Framework: To accurately identify which Android devices could reliably run AV1, Meta developed an ML framework. This system uses real-world low-level performance metrics (like `rtc_score`) to categorize device capabilities, allowing for scalable and accurate deployment decisions rather than relying on unreliable lab data or simple heuristics like memory/OS version.

Impact and Future Considerations

Through these systematic architectural and engineering efforts, Meta successfully expanded AV1 enablement to the majority of mobile devices in its RTC applications. This case study demonstrates that adopting advanced codecs requires a holistic system design approach, considering not just codec efficiency but also end-to-end latency, power consumption, application footprint, and scalable device compatibility mechanisms.