Title

AWS re:Invent 2022 - Accelerate deep learning & Habana Gaudi–based Amazon EC2 DL1 instances (PRT280)

Summary

• The session was presented by Greg Suraki, an applications engineer from Habana Labs, and Dvij Vachfai, a senior product manager at Amazon EC2.
• The focus was on the DL1 instances on EC2, which are built on Intel Habana Gaudi chips, designed specifically for machine learning training.
• The DL1 instances feature eight Gaudi chips per server, connected in a full mesh topology, with 400 Gbps networking bandwidth and 4 TB of local storage.
• The presenters emphasized the price-performance advantage of DL1 instances, which offer significant cost savings over traditional GPU instances for machine learning training.
• A live demo showcased the ease of migrating a PyTorch model to run on Gaudi using the Habana PyTorch library and the importance of the mark_step function for performance optimization.
• The session also covered TensorFlow model support and the collaboration between Amazon and Intel Habana to support a wide range of models and operators.
• The presenters highlighted customer use cases, including Mobileye for autonomous driving and Leidos for COVID-19 detection from X-ray images.
• The session concluded with a demonstration of distributed training on DL1 instances using PyTorch's distributed data parallel and MPIRUN.

Insights

• The DL1 instances are optimized for machine learning training, offering a balance between high performance and cost-effectiveness, which is crucial for scaling complex ML models and datasets.
• The integration of the Habana Gaudi chips into the DL1 instances demonstrates AWS's commitment to providing diverse hardware options for machine learning workloads, potentially lowering the barrier to entry for developers.
• The session highlighted the importance of seamless migration and compatibility with existing ML frameworks like PyTorch and TensorFlow, which is a key consideration for developers when adopting new cloud services.
• The use of distributed training and the ability to scale across multiple instances is indicative of the growing trend towards large-scale, distributed machine learning workloads in the cloud.
• The collaboration between AWS and Habana Labs, as well as partnerships with companies like Hugging Face, suggests a strong ecosystem developing around the DL1 instances, which could lead to broader adoption and innovation in the field of machine learning.