Agent Incident Registry catalogs 312 records spanning 2023-2026 with 41 percent realized harm in primary generative cases
The Agent Incident Registry supplies the first source-grounded catalog of agent failures with consistent labels for mechanism and outcome. It separates realized harm from demonstrations and supplies audit data against existing attack suites. The work establishes baseline criteria for agent incident reporting that future benchmarks and deployments can reference directly.
The registry addresses a documented gap in existing incident databases that omit tool-use chains and delegated-authority traces required for agent-security comparisons. Primary generative-system records show 41 percent realized harm concentrated in in-the-wild and safety-failure subsets, while responsible disclosures remain overwhelmingly demonstrated-only. A second reviewer validated all entries post-curation for label completeness.
InjecAgent's 30 cases map to three of AIR's twelve surfaces and are uniformly attacker-triggered, contrasting with AIR's 87 no-adversary safety failures. This distribution reveals that current public datasets over-represent demonstration failures relative to deployment incidents. The structure supports retrieval and scope auditing rather than rate estimation.
Standardized agent definitions embedded in the registry labels enable direct cross-evaluation with benchmarks such as WebArena and ToolBench failure traces. Prior incident efforts like the AI Incident Database captured 2022-2024 events without mechanism granularity; AIR adds surface and trigger fields that expose repeated tool-injection patterns across independent reports.
Next steps include quarterly ingestion of CVE-linked agent failures and integration with evaluation harnesses to test whether new agents reproduce registry mechanisms at scale.
AIR maintainers: at least 50 new records will be added by Q2 2027 with explicit tool-injection labels
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2609.11030)
- [2]Supporting Source(https://arxiv.org/abs/2307.02485)
- [3]Supporting Source(https://github.com/THUDM/InjecAgent)