Accelerating Large-Scale Test Migration with LLMs
Airbnb used an LLM-driven approach to migrate nearly 3.5K React component test files from Enzyme to React Testing Library (RTL). They initially estimated this would take 1.5 years but completed it in just six weeks. The migration involved refactoring test files while preserving the intent of the original tests and code coverage. They built an automated pipeline with validation, refactor steps, and retry loops. The LLM was given increasing context, including component code and related tests, to improve success rates. They used a "sample, tune, and sweep" approach to improve prompts and scripts, increasing completion from 75% to 97%. Automation efficiently handled the vast majority of the migration, leaving a small percentage for manual fixes. The project maintained test intent and code coverage and proved to be much faster and cheaper than a manual migration. This experience highlights the power of LLMs for large-scale code transformation. Airbnb plans to expand their use of LLMs for developer productivity. They are also hiring engineers who love solving complex problems at scale.