Microsoft's Cabra benchmark finds code understanding, not edit volume, is what trips up coding agents

Across 6,840 synthetic call-graph refactoring tasks, six agents in Copilot's harness stayed near-perfect by scripting around size with tools like grep, while plain LLMs degraded; on SWE-bench Verified traces, the count of read and analyze tool calls predicted failures better than lines edited. A new task that merges two near-identical classes finally broke the agents, and code plus data are open at github.com/microsoft/CABRA.