Title

AWS re:Invent 2022 - Cloud filesystem in studio workflows: A Netflix Drive case study (NFX203)

Summary

Netflix manages hundreds of petabytes of media assets and has developed a homegrown cloud file system called Netflix Drive.
Netflix Drive is a multi-OS, multi-interface cloud file system designed to provide seamless abstraction for object stores and supports a variety of data and metadata stores.
It is FUSE-based on Linux and macOS, and uses WinFSP on Windows, with an additional API interface for richer control.
The motivation behind Netflix Drive was to provide a common access plane for media assets, support legacy systems, and enable seamless and transparent movement of media assets through various stages of production.
Netflix Drive offers a rich control plane, standard file and folder access, and local file system performance, with the flexibility to adapt to different backend storage solutions.
Two case studies were presented: media workflow orchestrations (automated, programmatic use) and asset store use case (interactive use by artists on workstations).
The system allows for fine control over file access and synchronization, ensuring that artists and tools can operate with familiar interfaces and workflows while leveraging cloud storage capabilities.

Netflix Drive's architecture allows for a high degree of flexibility and control, which is crucial for managing the complex workflows associated with media production.
The system's API interface is particularly important for integrating with automated workflows and tools, allowing for programmatic control over file operations.
The ability to provide a POSIX-compliant interface on workstations is essential for compatibility with existing digital content creation tools and for minimizing disruption to artists' workflows.
Netflix Drive's design emphasizes the importance of performance, with local caching to provide near-local file system performance for demanding media applications.
The case studies demonstrate the system's versatility, catering to both backend automation and front-end user interaction, highlighting the importance of a file system that can adapt to different use cases within media production.
The use of cloud storage as a backend for Netflix Drive underscores the trend towards cloud-native solutions in media production, offering scalability, flexibility, and collaboration opportunities that traditional on-premises storage solutions cannot match.