When Should an AI Agent Stop? Note

When Should an AI Agent Stop?

The article proposes a progress-aware execution controller for agentic systems to address the inefficiency of arbitrary action limits. It argues that simply setting a max_steps for an agent does not guarantee task completion or prevent resource waste. Instead, a more intelligent system-level decision-making process is needed for stopping execution. The proposed architecture separates the agent from a controller that gates actions and evaluates progress.This controller tracks three key signals: task completion with evidence, actual progress to identify stagnation, and remaining budget. For instance, three searches yielding the same sources without new requirements being met would signal stagnation. The controller would then decide whether to finish, continue, replan, or stop incomplete. This rule-based policy would have distinct outcomes for each scenario.Importantly, the controller's state, including completed requirements and budget, must be durable to handle restarts. For an MVP, a small agent runtime with PostgreSQL for records and deterministic policy checks is suggested. The effectiveness of this approach would be tested by comparing it against fixed-limit and stagnation-aware policies.Metrics like verified task success, cost per successful task, latency, and premature stops would be used for evaluation. The goal is to save wasted work without compromising agent reliability. Ultimately, the next step for agentic systems is to build surrounding systems that intelligently decide when more work is useful. This architecture aims to enable more deliberate and efficient agent behavior, even if it means doing less.
CdXz5zHNQW_qPGfxFSYQb.webp