More Code Review = More Smelly... Note

More Code Review = More Smelly Code?

Sixty-eight review comments led to a function growing from 28 to 42 lines, with 62 fixes, each addressing a real problem. The issue was not the reviewer's accuracy, but the project's lack of a decision-making routine for comments. The AgentCoop project, an AI agent system, used OpenAI's Codex for automated code review, with a strict rule to address all comments. This often led to fixing comments without considering the broader implications.Four key problems emerged: scope creep from small, accurate fixes; urgent treatment of "security" or "availability" tags regardless of actual impact; poor trade-offs, where permanent code complexity was added for rare or non-existent issues; and fixing where the comment pointed, rather than the root cause. A "critical" issue concerning an upgrade breaking a config file was initially escalated but later resolved with a single paragraph in the documentation. The actual user impact was minimal, involving a few minutes of configuration adjustment, rather than a system-wide outage.The team then developed a routine to evaluate review comments, moving beyond gut feelings. First, three quick questions determine if a fix is cheap (under ten lines), if the issue can actually happen, and if it fails silently (requiring at least logging). Cheap fixes are implemented immediately, unreachable code paths are dropped, and silent failures are prioritized or made loud. The crucial caveat for "cheap" is to consider the cheapest effective fix, not necessarily the reviewer's suggestion.For comments surviving step one, a quantitative evaluation is performed in hours. This involves estimating the cost of the bug per incident, its annual frequency, the user's accepted pain (via a multiplier), the fix's build time, and its permanent annual maintenance cost. These values are used to calculate the annual savings and the payback period for the fix. If the net annual saving is zero or negative, the fix is deemed unworthy.The opening example, a config parsing issue, had a net negative saving when analyzed, proving it should not have been fixed. Estimating incident frequency requires building the number from specific, evidenced parts, rather than guessing. Race conditions are similarly evaluated by considering triggering actions and vulnerable windows, prioritizing scenarios where events are intentionally aligned over pure coincidence.