GUI grounding models that report over 85% accuracy on standard benchmarks drop 27-56 percentage points on spatial reasoning tasks per Sikka (2026).

The GUI-Perturbed framework introduces independent perturbations to visual scenes and instructions unlike single-screenshot benchmarks, revealing systematic accuracy collapse on relational instructions across three 7B models of the same lineage with 70% browser zoom producing statistically significant degradation (Sikka, 2026). Rank-8 LoRA fine-tuning on augmented data degraded performance rather than improving it, isolating failures in spatial reasoning, visual robustness, and reasoning calibration. Original abstract coverage missed the fine-tuning reversal and its diagnostic value for calibration issues (Sikka, 2026).

WebArena evaluations demonstrated parallel distribution-shift failures in web agents under realistic environments, with models exploiting superficial cues over true grounding (Zhou et al., 2023, https://arxiv.org/abs/2307.13854). SeeClick experiments similarly exposed naming shortcuts versus spatial understanding, a pattern extended by GUI-Perturbed perturbations that standard benchmarks overlook (Cheng et al., 2024, https://arxiv.org/abs/2401.10935). These connect to recurrent robustness failures in multimodal models under domain randomization.

As agentic systems increasingly rely on GUI grounding, the isolated axis failures indicate limited generalization beyond fixed benchmarks, providing targeted signals absent from aggregate metrics and highlighting the need for perturbation-based diagnostics in model development (Sikka, 2026; Zhou et al., 2023).