Pinterest's L1 ranking stage, crucial for ad performance, faced a persistent online-offline (O/O) discrepancy when deploying new conversion rate (CVR) models. Despite strong offline gains in loss and calibration, online A/B tests showed neutral or negative results, leading to launch delays. The investigation involved a full-stack diagnosis addressing model and evaluation, serving and features, and the funnel's impact. Initial checks ruled out offline evaluation issues, exposure bias, and serving failures as primary causes. Feature O/O discrepancy, where serving lacked features used in training, and embedding version skew, resulting in query and pin tower misalignment, were identified as key problems. The solution involved feature onboarding and addressing embedding skew, improving feature coverage and aligning model versions. Further analysis revealed the importance of funnel alignment and metric matching, where improvements in L1 metrics might not translate to CPA gains due to existing funnel limitations. This highlighted the need to consider O/O discrepancy as a core design constraint for model deployment.