The blog post discusses the technical insights into how Netflix's ML model serving infrastructure powers personalized experiences at scale across various domains. The central ML model serving platform exposes a domain-independent API abstraction and traffic routing capabilities to several domain-specific microservices for model inference. This singular API has increased the speed of innovation for iterating on newer versions of existing ML experiences and enabling new product experiences with ML. The success of the ML model serving infrastructure depends on enabling researchers to rapidly experiment with new hypotheses and safely release their models into production. The platform serves hundreds of model types and versions, netting 1 million requests per second, and operates at the level of workflows, not just individual scoring functions.The model definition contains a list of facts that it needs to compute features, and it relies on the model serving platform to supply these facts at serving time by calling several other microservices. The calling services only need to provide standard request context, and the relevant domain context, and the model can itself compute features and perform inference as part of the execution flow. The platform acts as an enabler of rapid ML innovation and limits the exposure of ML model iterations to the client apps. The key principles of the platform include model innovation independent of client apps, decoupling clients from model sharding, and flexible traffic routing rules.The platform uses a custom service called Switchboard, which serves as a flexible proxy layer for all traffic, handling over 1 million requests per second while maintaining high availability and reliability. Switchboard provides a single point of contact for all clients' model needs and can route a request based on a rich set of contextual features. The platform also introduces the concept of an "Objective", which is an enumeration defined by the serving platform that every request into the system must provide. The Objective decouples clients from concrete models and guides the platform's routing and model selection decisions.Switchboard Rules is a JavaScript configuration that allows researchers to attach model variants, experiments, and traffic splits to Objectives without changing client code. The rules dictate the default model to use for a given Objective, A/B experiments to configure for a set of Objectives, and customizations to gradually shift traffic to a new model. The rules are consumed by both Switchboard and the Model Serving clusters, and the serving platform components can take various actions based on these rules. Overall, the platform provides a scalable and flexible solution for serving ML models at Netflix, enabling rapid innovation and experimentation while minimizing the impact on client apps.