Title

AWS re:Invent 2022 - 3 innovations that redefine data protection for Amazon S3 (PRT315)

Summary

• Speakers: Woon Jung (CTO of Clumio), Peter Eaming (Principal Product Manager, Amazon S3), Mark Hoover (Senior Director of Engineering, Cox Automotive).
• Key Topics: Data protection for Amazon S3, focusing on backup, replication, high availability, durability, and availability.
• Amazon S3 Usage: Now used as primary production storage, storing over 280 trillion objects and handling over 100 million transactions per second.
• Data Protection Layers: Different layers of data protection are needed for different types of data stored in S3, including compliance data, user-generated data, and sensitive information.
• Risks and Solutions: Risks include accidental deletions, software errors, and malicious actions. Solutions include object versioning, Object Lock for immutability, and S3 replication for resiliency and compliance.
• S3 Storage Lens: A feature providing a central view of data protection capabilities across S3 buckets.
• Clumio's Role: Offers an independent and centrally managed backup solution that complements existing S3 data protection features.
• Cox Automotive's Journey: Adopted AWS well-architected framework, uses Clumio for data protection across 1,300 AWS accounts, and has developed a Terraform provider for integration with Clumio.
• Clumio Innovations: Announced support for 15-minute RPO (Recovery Point Objective), ability to handle up to 30 billion objects per bucket, and instant access to backup data through an S3-compatible endpoint.
• Live Demo: Showcased Clumio's backup and restore process, including the creation of protection groups, application of filters, and instant access to backup data.

Insights

• S3 as Primary Storage: The shift to using S3 as primary production storage for a wide range of applications, including data lakes and machine learning, signifies the need for robust data protection strategies.
• Data Protection Complexity: The complexity of protecting data at scale in S3 is highlighted, with the need for solutions that can handle billions of objects and provide granular control over what is backed up and restored.
• Clumio's Approach: Clumio's approach to data protection involves minimal footprint in customer accounts, leveraging AWS services like Lambda and EventBridge, and providing a dedicated AWS account per customer for data segregation.
• Cost Optimization: Clumio emphasizes cost optimization in data protection, allowing customers to specify what data to back up and restore, and offering features like instant access to reduce costs associated with traditional restore processes.
• Partnership and Integration: The partnership between Clumio and Cox Automotive demonstrates the importance of collaboration in developing data protection solutions that meet specific customer needs, including integration with existing AWS services and infrastructure.
• Operational Transparency: Clumio's internal workflow engine and observability tools provide operational transparency, enabling rapid troubleshooting and demonstrating a commitment to service reliability and customer trust.