Title

AWS re:Invent 2023 - Reducing AWS Fargate startup times by lazy loading container images (CON307)

Summary

AWS Fargate is a serverless compute engine for containers that provides task or pod level isolation and on-demand pricing.
Customers requested improvements in task startup times, scale, and performance for Fargate.
AWS has increased the number of concurrent tasks and the size of tasks that can run on Fargate.
The main bottleneck in task startup times was identified as the image pull and unpack stage.
AWS introduced lazy loading of container images using an open-source project called Seekable OCI (Sochi) to reduce startup times.
Sochi allows for downloading only the necessary parts of a container image, reducing the time and resources needed to start containers.
Fargate now supports Sochi, enabling tasks to start up to 60% faster without any changes to application code or ECS task definitions.
Customers such as Autodesk, Flywire, and Virtuso have reported significant improvements in task startup times and cost savings by using Sochi with Fargate.
A live demo showcased the impact of Sochi on Fargate, with lazy-loaded tasks starting 74% faster than tasks without lazy loading.

The adoption of Fargate for larger and more performance-sensitive workloads necessitated improvements in scalability and startup times.
The architecture of Fargate, which guarantees isolation for each task, made it particularly sensitive to image pull times, as each task runs on a fresh node and must pull all container images from a remote registry.
The introduction of Sochi as a lazy loading mechanism for container images is a significant innovation that addresses the specific challenge of slow task startups in Fargate.
Sochi's design allows for existing container images to be used without modification, and it integrates with containerd as an image management plugin.
The Sochi Index is a metadata file that provides an inventory of everything inside a container image, enabling the retrieval of only the necessary parts of the image.
AWS provides two open-source tools for building Sochi indices: the Sochi CLI for a manual approach and the Sochi Index Builder for an automated experience.
The use of Sochi is recommended for container images larger than 250 MB and may not be suitable for workloads that frequently access file system metadata or require a large portion of the container image.
The live demo and customer testimonials provide concrete evidence of the effectiveness of Sochi in improving Fargate task startup times, which can be crucial for time-sensitive and bursty workloads.