Title

AWS re:Invent 2022 - Run high-perf storage workloads on EC2 storage optimized instances (CMP312)

Summary

• Amit Shah and Shruti Gupta presented on optimizing storage workloads on AWS, focusing on EC2 storage-optimized instances and EBS.
• They defined storage-optimized workloads as those dealing with large data sets requiring high performance, throughput, and low latency, typically SQL/NoSQL databases, Kafka, Hadoop, and distributed file systems.
• AWS offers a variety of storage options, including block storage (EC2 instances and EBS), object storage (S3), file systems (EFS and FSx), and backup solutions.
• EC2 storage instances come with local NVMe SSDs for high performance and low latency, while EBS offers flexibility by allowing storage to scale independently of compute.
• AWS introduced Nitro SSDs in their fourth-generation products, offering lower latency and variability, and enabling enterprise-class storage features like Torn Write Prevention.
• The latest EC2 storage-optimized instances include Intel-based offerings (i3, i3en, i4i) and Graviton-based offerings (IM4GN and IS4GN), each with different performance and cost benefits.
• Strategies for migrating to fourth-generation instances include one-to-one mapping, scaling down, and scaling in, depending on performance and cost objectives.
• EBS offers SSD and HDD-backed volumes, snapshots, and data services, with the latest GP3 volumes allowing performance scaling independent of storage capacity.
• Choosing between Instant Store and EBS depends on performance, durability, and data protection requirements, with EBS recommended for higher durability and data protection needs.
• Specific examples of workloads were discussed, including Aerospike for Instant Store and SAP HANA for EBS.

Insights

• The introduction of Nitro SSDs represents a significant advancement in AWS's storage offerings, providing customers with improved performance and reliability.
• AWS's approach to storage optimization emphasizes flexibility, allowing customers to tailor their storage solutions to specific workload requirements.
• The ability to scale performance independently of storage capacity with GP3 volumes in EBS can lead to cost savings and more efficient resource utilization.
• The presentation highlighted the importance of understanding workload requirements in detail before choosing an instance type or storage solution, to ensure optimal performance and cost-effectiveness.
• AWS's continued innovation in storage solutions, such as the Torn Write Prevention feature, demonstrates a commitment to enhancing database performance and reducing total cost of ownership for customers.
• The comparison between Instant Store and EBS volumes suggests that AWS provides a comprehensive set of storage solutions to cater to a wide range of application needs, from real-time, high-performance workloads to persistent storage for mission-critical applications.