DEV Community
Follow
How I built a time-travel debugger for LLM agents on Burr, and what "replay the unchanged branch and prove nothing changed" actually takes.
Existing agent trace tools are limited, forcing users to re-run entire pipelines to debug issues, making it difficult to isolate changes from non-deterministic model behavior. Rewind addresses this by allowing users to fork a completed run at any step, modify a single input, and replay only the downstream steps. This enables precise diffing of trajectories, distinguishing genuine changes from model jitters. The Rewind architecture includes a browser interface, a FastAPI backend, and a Burr application, with state persisted in SQLite and telemetry captured per node.A key design principle is that state is semantic only, excluding non-deterministic elements like latency or token counts from the state itself. Framework-specific keys that would cause divergence are also stripped from the state before it impacts prompts or hashes. Overrides for forked runs are managed outside of the state, ensuring that unchanged branches can still hit the cache. Artifact hashes are generated from the content bytes, not the containing file path, preventing spurious diffs.The caching mechanism uses a deterministic key based on model, temperature, prompt, and tool result. Tool nodes have a separate cache keyed by tool name and canonical arguments. The diff engine flattens nested dictionaries to identify specific field-level divergences, providing clear and actionable debugging information. Rewind implements robust error handling, failing loudly on parse errors or unconsumable overrides.Verification tests confirm that Rewind achieves 100% cache hits and zero incremental token usage for unchanged replays. This determinism proves useful, as a run can survive a dead provider if all LLM calls are cached. Potential pitfalls include SDKs that mishandle metadata and inconsistent validation of model IDs by different providers. The tool also addresses common development environment issues like port contention and Vite's IPv6 defaults. Future enhancements include a "what-if" grid for parallelizing multiple overrides and support for non-linear graph topologies.