Scaling Media Processing with FFmpeg at Meta
Meta processes billions of video files daily using FFmpeg, presenting significant challenges for efficiency and reliability. This article details how Meta evolved its media processing architecture from a custom FFmpeg fork to leveraging upstream contributions for multi-lane transcoding and real-time quality metrics, significantly improving performance and enabling deprecation of their internal fork.
Read original on Meta EngineeringFFmpeg is a versatile, industry-standard tool fundamental to media processing at Meta, handling tens of billions of executions daily for tasks like video transcoding and editing. The scale and unique requirements of Meta's operations led them to maintain an internal fork of FFmpeg for many years, providing features like threaded multi-lane encoding and real-time quality metric computation that were not available in the open-source version.
The Challenge of Divergent Forks
The divergence of Meta's internal FFmpeg fork from the upstream version created maintenance burdens, including rebasing difficulties and the inability to easily leverage new codecs, formats, and reliability improvements from the open-source community. This necessitated a strategic shift towards aligning with upstream FFmpeg to streamline operations and benefit from community innovations.
Key Upstream Contributions for Scalable Transcoding
Meta collaborated with FFmpeg developers to upstream critical features previously found only in their internal fork. Two major advancements enabled this migration:
- Efficient Multi-Lane Transcoding: For Dynamic Adaptive Streaming over HTTP (DASH), multiple video encodings (different resolutions, codecs) are needed. Initially, this involved serial or parallel FFmpeg processes, leading to redundant decoding work. Meta's solution, now upstreamed into FFmpeg 6.0/8.0, enabled a single FFmpeg command to decode frames once and distribute them to parallel encoder instances, drastically reducing compute usage and improving efficiency for over 1 billion daily video uploads.
- Real-Time Quality Metrics: For livestreaming, real-time quality assessment is crucial. Meta's internal fork provided this by inserting in-loop decoders after each encoder to compare pre- and post-compression frames. This 'in-loop' decoding feature was integrated into FFmpeg 7.0, allowing real-time quality computation within a single FFmpeg command line.
Architectural Lesson: Upstream Where Possible
Meta's strategy highlights the benefits of contributing generalizable internal innovations back to open-source projects. This reduces maintenance overhead for custom forks, leverages community development, and ensures broader industry impact. Patches specific to proprietary infrastructure (like custom ASICs) are best kept internal.
These strategic contributions allowed Meta to fully deprecate its internal FFmpeg fork for VOD and livestreaming pipelines, consolidating their media processing infrastructure around a single, robust, and community-supported open-source tool. The article also mentions the integration of custom hardware (Meta Scalable Video Processor - MSVP) through FFmpeg's standardized hardware APIs, minimizing platform-specific quirks while keeping the ASIC-specific patches internal.