Title

AWS re:Invent 2023 - Automate the modernization of legacy ETL to AWS Glue using LeapLogic (ENT328)

Summary

AWS Glue is a serverless data integration service that is cost-effective, scalable, and supports open-source engines like Spark and Python.
AWS Glue has been enhanced with new connectors for various data sources, including Iceberg, Delta, Apache Hudi, Redshift, BigQuery, Snowflake, Teradata, MongoDB, Vertica, Amazon OpenSearch, SAP HANA, Azure Cosmos DB, and Azure SQL Database.
AWS offers native connectors, marketplace connectors, and custom connectors through exposed APIs for data ingestion.
An ETL modernization program was launched to help customers migrate to AWS Glue, offering a no-cost assessment and proof of concept for legacy ETL code conversion.
LeapLogic, a product accelerator by Impetus, can achieve up to 80% code conversion, aiding in the migration process.
Trends in the field include a shift away from legacy ETL solutions, cost optimization by decoupling ETL from data warehousing, adoption of AWS Glue for scalability and flexibility, and preference for open-source ETL solutions.
LeapLogic uses a machine learning engine to analyze and convert legacy ETL code to AWS Glue, offering a fixed bid and duration for projects.
The LeapLogic process includes assessment, transformation, validation, and operational phases, ensuring code functionality and performance.
Impetus partners with AWS professional services and is MAP certified, offering demos and assessments at their booth.

The shift towards serverless data integration services like AWS Glue is driven by the need for cost efficiency, scalability, and flexibility in managing data workloads.
The expansion of AWS Glue's connectors demonstrates AWS's commitment to interoperability and the ability to handle diverse data sources, which is crucial for modern data analytics.
The partnership between AWS and Impetus, leveraging LeapLogic, indicates a growing ecosystem around AWS services, where third-party solutions are playing a significant role in facilitating the migration and modernization of legacy systems.
The emphasis on automation in the modernization process, as highlighted by the success of LeapLogic, suggests that there is a significant demand for tools that can reduce manual effort and accelerate migration timelines.
The trends observed by AWS analytics principal Grace Tonsich reflect a broader industry movement towards cloud-native solutions and the decoupling of ETL from data warehousing to achieve cost savings and performance improvements.
The detailed explanation of LeapLogic's capabilities and methodology provides insight into how complex legacy migrations can be managed and executed with predictability in terms of time and cost, which is a key concern for enterprises undertaking such transitions.