Existing evaluations rely on low-fidelity environments, simulated APIs, or narrowly scoped tasks. BeSafe-Bench covers four domains: Web, Mobile, Embodied VLM, and Embodied VLA. It constructs a diverse instruction space by augmenting tasks with nine categories of safety-critical risks using functional environments. (arXiv:2603.25747)

A hybrid evaluation framework combines rule-based checks with LLM-as-a-judge reasoning to assess real environmental impacts. 13 popular agents were evaluated. (arXiv:2603.25747)

The best-performing agent completes fewer than 40% of tasks while fully adhering to safety constraints. Strong task performance frequently coincides with severe safety violations. (arXiv:2603.25747)