Android end-to-end testing builds at Pinterest were slow and unreliable due to unbalanced test shards and platform limitations. The team first evaluated third-party solutions but found them inadequate for their needs. They decided to build an in-house testing platform called PinTestLab, hosted on EC2 emulators. This platform allowed for complete control over the testing stack and infrastructure.The core innovation is a runtime-aware sharding mechanism. This system uses historical test duration and stability data to pack tests into shards. The goal is to ensure that each shard has a similar total runtime. This approach differs from simply balancing the number of tests per shard.Previously, package-based sharding led to imbalances where a single slow shard would delay the entire build. Even simple time-based sorting failed to account for emulator idle time. The new runtime-aware sharding algorithm works by sorting tests by average runtime and then greedily assigning each test to the emulator projected to finish earliest. This keeps all emulators busy and minimizes the time difference between the fastest and slowest shards.The impact of this solution has been significant. End-to-end build times were reduced by nine minutes, a 36% improvement. The runtime of the slowest shard decreased by 55%. The time difference between the fastest and slowest shards was dramatically compressed from 597 seconds to just 130 seconds. This boosts developer velocity by providing faster and more reliable feedback.