Title

AWS re:Invent 2022 - S&P Dow Jones Indices’ AWS journey to Amazon Aurora (DAT319)

Summary

• Saikath Banerjee, a senior database specialist solutions architect, and Kumar Bangalore, a senior director of database engineering from S&P Global, presented S&P Dow Jones Indices' modernization journey to Amazon Aurora.
• S&P Dow Jones Indices is a segment of S&P Global, known for calculating real-time market performance indices like the S&P 500 and Dow Jones Industrial Average.
• The company's application landscape includes platforms for core index services, fixed income, derivatives, commodities, and index data services.
• They migrated from on-premises databases to AWS, initially using EC2 instances, then moving to Amazon RDS and finally to Amazon Aurora.
• The migration involved re-hosting, re-platforming, and modernizing applications, with a focus on automation and cloud-native capabilities.
• Kumar shared the challenges and solutions encountered during the migration, particularly with schema conversion and database migration using AWS DMS.
• They leveraged AWS services like CloudFormation, AWS Backup, Database Activity Stream, SailPoint, CyberArk, Datadog, CloudWatch, and Performance Insights to improve efficiency and reduce operational costs.
• The session concluded with insights on Amazon Aurora's architecture, features, and migration strategies, including best practices for using DMS and SCT.

Insights

• S&P Dow Jones Indices' migration to Amazon Aurora was driven by the need for real-time transactional capabilities, historical data management, and confidentiality of non-public information.
• The phased migration approach allowed for gradual modernization and minimized disruptions to their services.
• The migration to Amazon Aurora provided benefits such as managed database services, high availability, read replicas, cross-region disaster recovery, and reduced operational overhead.
• Challenges during the migration included schema conversion issues, performance regression with prepared statements, and large-scale data migration logistics.
• AWS Infrastructure Event Management (IEM) team provided crucial support during the migration, offering guidance and solutions to overcome challenges.
• The use of AWS services and automation tools significantly improved operational efficiency and allowed for better resource management.
• The presentation highlighted the importance of choosing the right migration path based on application needs and the potential for parallel execution of migration and modernization processes.
• Amazon Aurora's architecture, with its distributed storage across three AZs, write quorum, and read replicas, offers a robust solution for high availability and scalability.
• The session underscored the importance of observability, security, and auditing in database management, particularly in the financial services industry.
• The talk emphasized the continuous journey of cloud adoption and innovation, with S&P Dow Jones Indices planning to leverage more AWS components in the future.