Trail of Bits Blog
Follow
Your AI agent fixed the bug, but what else did it change?
Trail of Bits has released two new plugins, patching-bugs and regression-hunt, designed to automate patch development and regression checks for engineers. The patching-bugs plugin identifies bugs, creates fixes, and tests them, while regression-hunt verifies that code changes do not break existing functionality. These tools are a product of the Patch the Planet initiative and draw from extensive analysis of security patches and engineer sessions. They offer maintainers a structured process, evidence, and best practices to enhance their patching efficiency. Preliminary evaluations show regression-hunt reduces false positives significantly compared to a basic AI agent.The company implemented these plugins after observing regressions in their own security patches, realizing the need for standardized approaches to both patch creation and regression detection. The plugins incorporate modular steps, strict input/output schemas, and live proof-of-concept executions for every claim. Regression-hunt defines regressions rigorously and incorporates threat models to ensure thoroughness, resulting in fewer false positives at the cost of some borderline cases. Patching-bugs distills best practices from thousands of AI sessions, emphasizing a comprehensive three-arm proof and invariant-guided repair.Future development will focus on evaluating patching-bugs, learning from agent traces, and increasing configurability. To maximize their effectiveness, the plugins require project-specific knowledge like a THREAT_MODEL.md and PATCHING_GUIDELINES.md. The industry increasingly needs AI-driven solutions to accelerate the bug-fixing process, balancing speed with quality, and these plugins aim to contribute to that goal. Developers are encouraged to install and use the plugins, provide project context, and share feedback to shape future versions.