Round 2: A Survey of Causal In... Note

Round 2: A Survey of Causal Inference Applications at Netflix

At Netflix, causal inference is crucial for decision-making and improving member experiences. The annual Causal Inference and Experimentation Summit brings together experts to share methodological developments and innovative applications.One talk focused on developing a faster, automated approach to estimate annualized impact from A/B testing, addressing challenges with unobserved billing periods and unobserved sign-up cohorts.Another talk presented a systematic framework for evaluating game events using synthetic control models, handling limited data and intervention scenarios where A/B testing is impractical.Double Machine Learning was applied to compare the importance of different metrics in A/B tests, addressing potential biases when treatment effects are heterogeneous.Survey AB tests with heterogeneous non-response bias were analyzed using conditional average treatment effects, propensity scores, and iterative proportional fitting to ensure accurate representation of member opinions.Design was highlighted as an integral part of causal inference at Netflix, with a focus on optimizing user interfaces and data presentation to facilitate decision-making through experimentation.