VentureBeat
Follow
Agentic reliability and evaluations : Enterprises that got burned by a bad eval are the most likely to remove humans from the loop, not the least
In July, enterprises reported a sharp increase in trust towards automated agent evaluation systems. The share of organizations fully trusting these systems nearly tripled from June to July. Concurrently, the perceived mismatch between evaluations and real-world outcomes decreased. However, the actual failure rate of agents that pass evaluations but then fail customers remained unchanged. This high failure rate continues to affect just under half of all organizations surveyed. The increased trust is primarily driven by enterprises that have not yet experienced customer-facing failures from their AI agents. Organizations that have encountered these failures demonstrate significantly lower trust in automated evaluations. Interestingly, experiencing such failures accelerates the adoption of agent autonomy rather than slowing it down. The vendor market for evaluation tooling is consolidating, with specialist platforms gaining traction. Enterprises are increasingly prioritizing ease of integration over cost when selecting evaluation tools.