Unwritten Benchmark: GPT-4o and Gemini 2.5-Pro score below 10% on acousto-kinematic inference

The arXiv paper 2608.14558 defines acousto-kinematic word inference across three writing styles with no visible ink. Models receive only synchronized audio of scratches and video of hand trajectories. Evaluation covers GPT-4o, Gemini 2.5-Pro and prior multimodal baselines. All systems remain below 10% letter accuracy while human subjects reach 80%+ ordered accuracy.

Results document a fusion penalty: joint audio-video input lowers accuracy relative to the stronger single modality for every tested model. This pattern contradicts standard multimodal scaling expectations documented in the GPT-4o system card and the Gemini technical reports. The gap isolates a specific deficit in micro-kinematic causal inference rather than generic perception.

The benchmark itself supplies an open evaluation harness that any lab can run without proprietary data. Prior open perceptual suites such as Ego4D and AudioSet measured recognition of visible or audible content; this task requires reconstruction of an unseen generative process. Public release therefore supplies a concrete, reproducible stress test for next-generation cross-modal architectures.

Subsequent work will likely target explicit kinematic simulation layers or causal world models to close the observed gap. Any model claiming general multimodal reasoning must now report numbers on this task within the next two release cycles.