Title

AWS re:Invent 2022 - Diving deep with Amazon Ads: Analytics at scale (AMZ303)

Summary

Varun Kamlakarn, a principal customer solutions manager at AWS, introduces the session on analytics at scale by Amazon Ads.
Tom Skinner, director at Amazon Ads, discusses the importance of analytics in advertising, the challenges faced, and the metrics for success.
Josh Angel, a principal engineer at Amazon Ads, delves into the technical solutions for orchestration, data applications, and reporting.
Amazon Ads handles over 100 petabytes of data, generates 40 billion reports annually, and processes over a trillion events weekly.
The team has developed a concept called "data rivers," which is a hybrid of data lakes and data streams, to process data efficiently.
The session covers the evolution of Amazon Ads' data processing from data warehouses to data lakes to data rivers.
The reporting capability of Amazon Ads is robust, handling trillions of events and petabytes of data with high throughput and low latency.
The speakers invite attendees to meet them at the Amazon Ads booth and request feedback on the session.

Amazon Ads has achieved significant scale in data processing and reporting, which is critical for providing accurate and timely information to advertisers.
The concept of "data rivers" is a novel approach that combines the flexibility of data lakes with the real-time processing capabilities of data streams, optimized for Amazon Ads' specific use cases.
The transition from monolithic data warehouses to more flexible and scalable data lakes, and eventually to data rivers, highlights the importance of evolving data architectures to meet growing demands.
The use of AWS services like EMR, DynamoDB, S3, Glue, Athena, OpenSearch, EKS, Lambda, and Fargate demonstrates the versatility and integration capabilities of AWS for building complex, large-scale data processing systems.
The session emphasizes the importance of separating storage from compute, avoiding monolithic clusters, choosing the right technology for the task, and processing data in smaller batches for improved performance and operational efficiency.
The detailed breakdown of Amazon Ads' technical journey provides valuable insights for other organizations facing similar challenges in data processing and analytics at scale.