Netflix handles millions of daily workloads, many of which fail, creating a significant debugging burden for engineers. Their initial solution, Pensive, used predefined rules to auto-diagnose and offer remediation for about half of these failures. However, this rule-based system struggled with unknown failure modes and required constant manual updates.To address these limitations, Netflix developed Pensive AI, integrating artificial intelligence to enhance error extraction and diagnosis. The goal was to provide broader coverage, lower latency, and controlled AI costs within the existing development lifecycle. Pensive AI wraps AI capabilities around deterministic processes, improving maintenance and expanding coverage.Early results show Pensive AI covers 100% of failures, including previously unknown ones. Manual debugging actions decreased significantly when Pensive AI was used. The system features two complementary paths: always-on error extraction for a quick initial assessment and on-demand debugging for deeper analysis using LLMs, triggered when the first look is insufficient.Error extraction identifies key error spans within logs by returning line numbers rather than full text, improving efficiency and reducing hallucinations. This automated process complements existing rule-based error detection within strict timeouts for timely responses. The system also manages data contracts carefully, introducing a new model alongside the old one to allow gradual adoption.Pensive AI Debugging offers root-cause analysis and remediation suggestions for failed step attempts, accessible via APIs and user interfaces. This on-demand debugging, which incurs higher costs, is initiated only when necessary to manage expenses effectively. Debugging responses are cached per step attempt to avoid redundant computations and reduce latency.The system streams responses as they are generated, providing a live-feeling experience for users even if connections are interrupted. Generated responses persist even if the requesting client disconnects, ensuring the work is not lost. This design is independent of the connection that initiated it, allowing for robust and continuous debugging.Netflix also developed separate evaluation frameworks to ensure the reliability of Pensive AI's error extraction and debugging guidance. For extraction, existing hand-written rules serve as a benchmark for comparison without requiring extensive manual relabeling. These frameworks help maintain performance on failures that the legacy Pensive system already handled.