Title

AWS re:Invent 2023 - AWS Storage for Serverless Application Development (STG311-R)

Summary

The session focused on AWS storage solutions for serverless application development, emphasizing the importance of data accessibility in serverless environments.
Brian Lyles, a senior principal engineer at Amazon S3, and Sebastian Baruba, a manager of specialist solution architecture at AWS, presented the session.
Jefferson Frazier from Shutterstock shared a customer use case, detailing their serverless architecture for content ingestion and delivery.
The talk covered serverless compute options (Lambda and Fargate), integration services (EventBridge, Step Functions, SQS, SNS, API Gateway, AppSync), and storage services (S3 and EFS).
The presenters discussed the benefits of serverless, including rapid scaling, cost savings, and focus on building applications rather than managing infrastructure.
They explained the differences between S3 and EFS, with S3 being suitable for object storage with high parallelism and EFS for file storage with POSIX compliance.
Real-world use cases were presented, including dbrain AI's use of EKS with EFS and S3 for scalable AI video creation, and Woot's use of S3 for modernizing their data warehouse.
Shutterstock's use case highlighted their serverless architecture for content ingestion and delivery, leveraging EventBridge, Step Functions, and S3, resulting in significant improvements in operational velocity.

Serverless architectures are increasingly being adopted for their ability to handle unpredictable workloads and scale automatically, reducing operational overhead and costs.
AWS provides a comprehensive set of services that support serverless compute and storage, allowing developers to choose the right tools based on their application needs.
The distinction between object storage (S3) and file storage (EFS) is crucial, with each serving different use cases and performance requirements.
Real-world examples from companies like dbrain AI and Woot demonstrate the practical benefits of serverless architectures in handling high-demand, data-intensive applications.
Shutterstock's experience underscores the importance of designing serverless systems with strong prefix entropy, transparent bucket naming conventions, and attribute-based access controls to manage a large and growing storage footprint effectively.
The session highlighted the importance of focusing on the smallest repeatable processes when transitioning from legacy systems to serverless, allowing teams to concentrate on delivering customer value rather than infrastructure management.
AWS training badges for storage are available for those interested in deepening their knowledge of AWS storage offerings, indicating AWS's commitment to education and skill development in the cloud storage domain.