Title

AWS re:Invent 2023 - Best practices for moving data to AWS using online and offline methods (STG206)

Summary

• Jeff Bartley and Eric Durand, Product Managers for AWS DataSync and Snow Family respectively, presented on data migration services provided by AWS.
• Key business drivers for data migration include retention and compliance, business continuity, cost reduction, and capacity growth.
• DataSync is an online data transfer service that uses a custom protocol for fast, secure, and verified data transfer, supporting NFS, SMB, and S3 API compatible storage systems.
• DataSync use cases include accelerating recurring data transfer workflows, migrations, ongoing replications for business continuity, and archiving cold data.
• DataSync can move data to/from on-premises, edge, and multi-cloud environments, and between AWS storage systems, with all traffic encrypted and within the AWS network.
• Snow Family is an offline data transfer service using physical devices (Snowcone and Snowball Edge) for moving large volumes of data securely and efficiently.
• Snow Family devices are rugged, tamper-resistant, and support a variety of AWS services for edge computing.
• OpsHub is used to manage Snow devices, allowing non-technical personnel to perform data collection and migration.
• The Snow Large Data Migration Manager is a tool for planning and monitoring large-scale data migrations.
• DataSync and Snow can be used together for initial bulk data transfer (offline) and subsequent incremental transfers (online).
• The choice between online (DataSync) and offline (Snow) methods depends on speed requirements, bandwidth availability, and data set characteristics.

Insights

• DataSync is suitable for large-scale migrations, especially when speed and security are critical, and for environments with sufficient bandwidth.
• Snow Family is ideal for environments with limited or no network connectivity, or when physical shipment of data is more practical.
• The introduction of the denser Snowball Edge Storage Optimized device indicates AWS's commitment to facilitating large-scale data migrations.
• The use of both DataSync and Snow Family services together suggests a hybrid approach to data migration, leveraging the strengths of both online and offline transfer methods.
• AWS emphasizes the importance of security in data transfer, with both DataSync and Snow Family offering robust encryption and data protection features.
• The presentation underscores the need for businesses to evaluate their data migration strategies carefully, considering factors such as data type, urgency, and infrastructure limitations.
• AWS provides a variety of tools and resources, including the Snow Large Data Migration Manager and OpsHub, to simplify the data migration process and enable customers to focus on their core business activities.