Title

AWS re:Invent 2023 - Netflix's success: Combining collaboration, hardware monitoring & AI (AIM315)

Summary

Speakers: Harshad Sane (Principal Engineer at Intel) and Vadim Filanovsky (Performance Engineer at Netflix).
Collaboration: Intel and Netflix have been collaborating to optimize performance on AWS, focusing on hardware monitoring and AI usage.
Hardware Monitoring: Intel provides software optimizations and contributes to open-source software. Netflix runs on Intel Xeon processors on AWS and benefits from Intel's expertise across the hardware stack.
Observability at Netflix: Netflix uses a three-level observability approach: infrastructure, service, and instance levels. They encountered a problem with bimodal CPU distribution during a migration, which was later identified as false sharing in the JDK.
Intel's Methodology: Intel uses a step-by-step methodology for diagnosing performance issues, including hardware characterization with PMUs and software profiling with tools like Intel VTune Profiler.
AI Usage at Netflix: Netflix uses AI in its encoding pipeline, particularly for downsampling video content to optimize for different network conditions and device capabilities. They've seen significant performance improvements using Intel's 1DNN library on Intel Xeon processors.
Future Collaboration: Netflix looks forward to further collaboration with Intel on Sapphire Rapids processors to continue improving performance and efficiency.

False Sharing Issue: Netflix encountered a false sharing problem within the JDK during a migration to a larger AWS instance type. This issue was causing significant performance degradation and was resolved by modifying the data layout in the JDK, leading to a 3.5x improvement in throughput.
Intel's Hardware Innovations: Intel's PMU and AMX technologies provide deep insights into CPU performance and enable significant optimizations for AI workloads. The introduction of AMX on Sapphire Rapids processors offers potential for further performance gains.
Netflix's Encoding Pipeline: Netflix's use of AI in its encoding pipeline demonstrates the importance of optimizing for perceived quality over raw bitrates. Their approach to downsampling using a neural network improves the quality of experience for users, especially in regions with limited bandwidth.
Importance of Software Optimization: The collaboration highlights the critical role of software optimization in maximizing hardware performance. Intel's investment in software and its contributions to open-source projects are key differentiators that benefit customers like Netflix.
Resource Utilization: Netflix's strategy of using general-purpose hardware for its encoding pipeline and repurposing unused capacity for encoding tasks underscores the importance of resource efficiency and the benefits of flexible cloud infrastructure.