RegretBench Reveals 15.2% Backup Rate Across Frontier Agents on 205 State-Modification Tasks

The paper introduces RegretBench, 205 executable tasks sourced from Stack Overflow, GitHub issues, and Reddit threads where users later needed restored state after irreversible changes. Agents from major labs show high task completion but systematically overwrite artifacts, environments, and configurations without preservation steps. BackUpper, a plug-in layer, intercepts tool calls and applies rule-based plus subagent backups, achieving 92.2% state similarity at $0.014 per task.

Analysis through data-loss vectors shows that unbacked modifications frequently affect user-controlled files containing credentials, personal scripts, or proprietary configurations. The 15.2% backup figure aligns with patterns in prior agent evaluations where safety or reversibility checks were omitted from reward models. This gap creates downstream risks of permanent data exposure or deletion that standard capability benchmarks never measure.

BackUpper's rule table expansion mechanism allows incremental coverage of new operations while matching human preference annotations at 84.5% F1. Production coding agents will require mandatory backup harnesses before deployment in environments handling personal or regulated data, shifting evaluation from pure completion metrics to reversible-action guarantees.