AWS Big Data Blog
Follow
Upgrade AWS Glue jobs to Glue 6.0 with AI-powered Spark upgrades
Walk through upgrading a PySpark ETL job from AWS Glue 5.1 to AWS Glue 6.0 using the generative AI upgrades for Apache Spark. The upgrade analysis automatically detects incompatibilities, applies fixes, and validates results with data quality checks.