iFLYTEK-Embodied-Omni Integrates VLM VGM AGM via Shared Multimodal Self-Attention

The June 2026 arXiv report details a unified architecture that replaces cascaded vision-language to video to action pipelines. Modality-specific encoders feed into shared self-attention layers enabling direct communication between instruction parsing future-state prediction and low-level motor chunk generation. The design labels the VLM and VGM as brain for planning and the AGM as cerebellum for execution.

Training combines human demonstration robot interaction and general image-text corpora. The four-stage schedule first aligns the VLM then the VGM then the AGM before end-to-end joint fine-tuning. No per-component benchmark scores appear in the submission yet the authors claim reduced interface bottlenecks relative to prior modular stacks.

The approach mirrors patterns in RT-2 and OpenVLA where joint training on mixed embodied data improved zero-shot generalization. By keeping video prediction and action heads inside one attention graph the model avoids compounding errors at module boundaries that have limited earlier cascaded agents on long-horizon tasks.

Operational deployment will require public release of the checkpoint and evaluation on standardized suites such as RLBench or BridgeData V2 within the next nine months to confirm claimed gains over separate VLM-plus-controller baselines.