Title

AWS re:Invent 2022 - Best practices for moving data to AWS, online and offline (STG210)

Summary

• Randy Boutin, Miriam Belhadj, and Eric Durand presented best practices for moving large amounts of data to AWS.
• They discussed the challenges of large-scale data transfers, such as managing scripts, ensuring data security, and handling network errors.
• AWS offers a portfolio of services for data migration, including AWS DataSync for online transfers and AWS Snow Family for offline.
• Key considerations for choosing between online and offline methods include available bandwidth, firewall configurations, and immediate data availability.
• AWS DataSync is a managed service that facilitates fast, secure, and cost-effective online data transfers.
• AWS Snow Family provides physical devices for offline data transfer, suitable for environments with limited or no network connectivity.
• AWS DataSync Discovery is a new feature that helps plan and implement data migrations by analyzing on-premises storage usage and performance.
• AWS DataSync agent is deployed on-premises for data transfer, with various connectivity options including private connections, public endpoints, and FIPS endpoints.
• AWS Snow Family devices are rugged, secure, and designed for both offline data movement and edge computing.
• Snow devices support a range of AWS services and are managed through OpsHub, a graphical user interface.
• Large Data Migration Manager is a tool for planning and managing large-scale data transfers using multiple Snow devices.
• Combining AWS DataSync and Snow Family can be effective for certain use cases, such as moving critical application data with minimal downtime.

Insights

• The choice between online and offline data migration methods is influenced by specific project requirements, such as bandwidth availability, data sensitivity, and timeline constraints.
• AWS DataSync's cost-effectiveness and ease of use make it a compelling option for customers looking to migrate applications and cold data to the cloud.
• The AWS Snow Family's rugged design and secure data handling capabilities make it suitable for data transfers in challenging environments and for customers with large-scale data center migrations.
• AWS DataSync Discovery can significantly aid customers in understanding their current storage systems and making informed decisions about which AWS storage services to use.
• The introduction of the Large Data Migration Manager indicates AWS's commitment to streamlining the data migration process for customers with extensive data transfer needs.
• The use of AWS DataSync in conjunction with Snow Family devices for incremental data transfer after an initial bulk copy demonstrates AWS's flexibility in accommodating complex migration scenarios.
• Customer examples, such as T-Map Mobility and Join, provide real-world evidence of the effectiveness of AWS's data migration services in meeting tight deadlines and reducing operational costs.