Scaling Etsy Payments with Vitess: Part 2 – The “Seamless” Migration

Etsy migrated 40 billion rows from unsharded databases to a Vitess-managed, sharded environment. The migration involved redistributing data, verifying completeness, and switching reads/writes. Staging was crucial for testing and building confidence in the process. VReplication enabled smooth read/write switching and data consistency. Unexpected query volume increase was addressed with CreateLookupVindex, preventing scatter queries. Secondary Vindexes were applied during the migration to optimize shard identification. Due to Vindex maintenance limitations, writes were switched back to the source after externalizing Vindexes. Destination cluster specs were upgraded to handle the load after externalizing Vindexes. The migration was completed without disruption or downtime, thanks to extensive testing and careful planning. This was part 2 of a series on sharding Etsy's payments data with Vitess.