Title

AWS re:Invent 2023 - What’s new with Amazon EMR and Amazon Athena (ANT204)

Summary

Vinita Anand, product lead for EMR and Athena, introduced new features and capabilities developed based on customer feedback.
Mohammad Rehan from Telecom Cell shared how his company leveraged AWS for operational data modernization.
Radhika Ravirala, principal product manager, discussed deployment options and security features.
Key themes included AI, data analytics, cost reduction, performance optimization, and security.
EMR now supports the latest open-source frameworks and open table formats like Apache Iceberg, Hudi, and Delta.
EMR runtime has been optimized for better performance, especially with Graviton3 processors.
EMR Serverless and EC2 have new features, including support for Graviton2, custom images, and improved cost visibility.
EMR on EKS allows running open-source frameworks on Kubernetes, with new features like dynamic pod auto-scaling and Apache Flink support.
Athena introduced provisioned capacity for better control over workloads, cost-based optimizer for performance, and support for S3 Express One Zone for faster queries.
Security enhancements include native LDAP integration for EMR and Trusted Identity Propagation for end-to-end auditability.

The integration of EMR and Athena teams within AWS signifies a strategic move towards unification and simplification of data services.
AWS's commitment to supporting the latest open-source frameworks within 90 days (30 days for popular ones) indicates a strong focus on keeping their platforms up-to-date with community developments.
The emphasis on cost reduction and performance optimization reflects AWS's response to customer demands for more efficient and cost-effective data processing solutions.
The adoption of Graviton3 processors and the support for S3 Express One Zone demonstrate AWS's continuous innovation in hardware and storage solutions to improve performance and reduce costs.
The introduction of serverless options and the expansion of deployment models cater to the growing demand for flexible, scalable, and easy-to-manage data processing environments.
The focus on security, with features like fine-grained access controls and identity propagation, highlights the increasing importance of data governance and compliance in cloud services.
The case study of Telecom Cell's successful migration to EMR illustrates the practical benefits of AWS's big data analytics solutions in a real-world scenario, showcasing significant improvements in performance, cost, and operational efficiency.
The ongoing improvements to the user experience of EMR, Athena, and other AWS data analytics services indicate a commitment to making these tools more accessible and user-friendly.