The So-fine Real-time ML Paradigm

Etsy's annual CodeMosaic hackathon showcased an ambitious project to develop a system for stateful machine learning training and online learning. The team aimed to update model weights incrementally, saving costs and improving metrics. On Day 1, they planned their approach, dividing into subteams focused on real-time training data, a learning service, and evaluation. Day 2 saw the teams implementing their plans, with challenges including data formatting and model selection. The evaluation team proposed a competition to compare the performance of a continuously trained model against a batch-trained model. Day 3 brought presentations and discussions of potential impact, including an estimated $212K annual cost saving for Ads models alone. The team acknowledged the hurdles to productionizing the system, including infrastructure and quality assurance, but expressed optimism for the future of complex ML architectures.