Title

AWS re:Invent 2022 - Modernize and gradually migrate your data model from SQL to NoSQL (BOA321)

Summary

The session, led by AWS Solution Architect Alex Schuren, focused on the modernization journey of migrating data from SQL to NoSQL databases.
Alex discussed the challenges faced by companies with legacy systems and the need for modernization to improve performance and innovation.
He introduced the concept of the modernization journey, which includes discovery, assessment, and deciding whether to repurpose, retain, or retire applications.
The session covered mechanisms for migration, including Change Data Capture (CDC) and dual writes, and the trade-offs associated with each method.
Alex emphasized starting with the customer in mind to understand the value of specific data for applications and systems.
The testing phase was discussed, highlighting the importance of validating data consistency and handling failures.
A live demo was conducted to demonstrate the migration process, using AWS CDK, feature flags, and observability tools.
The session concluded with a Q&A segment.

Modernization is not just about building new systems but also about improving existing ones, which can be more challenging but also more rewarding.
The migration from SQL to NoSQL is not a one-size-fits-all solution; it requires a tailored approach based on the specific needs and constraints of the customer.
Change Data Capture (CDC) is a database-level approach that does not require code changes and is suitable for large volumes of data, but it can be complex and requires additional infrastructure.
Dual writes are an application-level approach that allows for easy data segmentation and requires less infrastructure, but it can introduce data inconsistency and complexity in the code.
Feature flags are a powerful tool for controlling the scope of migration and for gradually shifting from old to new systems without disrupting the entire application.
Observability and monitoring are crucial for gaining confidence in the migration process, ensuring data consistency, and identifying issues early on.
The modernization journey is ongoing, and systems will continuously need to evolve to adapt to new technologies and business requirements.