ZGCM-1 7B records 4.2x 16K pre-training speedup via FP8 Muon and gated sliding-window attention

The arXiv 2609.13356 release includes full weights from pre-training, mid-training, and post-training stages plus intermediate checkpoints, training code, per-stage datasets, and W&B logs. Training used interleaved gated sliding-window plus full attention, a stable FP8 Muon optimizer, and progressive context scaling through 16K, 64K, and 256K stages. Agent swarms handled cluster operations, data curation, and rapid diagnostics throughout the lifecycle.

Benchmark results show ZGCM-1-7B competitive with the 7B model class on general tasks and within striking distance of models 30x larger on math and agentic suites. The reported 4.2x reduction in 16K pre-training time-to-loss is measured against an internal baseline without the co-designed attention and optimizer stack. Eight empirical findings cover architectural scaling limits, SFT pruning thresholds, long-context generalization curves, and agentic co-training dynamics.

The core premise—that compact models must couple internal reasoning with external tools rather than memorize the web—directly extends prior work on tool-augmented reasoning in DeepSeek-V3 and Qwen2.5-Math. By casting interaction traces as Markov Decision Processes, the mid-training stage creates verifiable credit assignment absent in standard SFT pipelines. This approach trades parametric capacity for verifiable external search steps, a pattern already visible in OpenAI o1 and Anthropic tool-use traces.

Operational impact centers on the released artifacts: downstream labs can now replicate the 256K MDP curriculum and FP8 Muon stack without starting from random initialization. The open data recipes and checkpoints enable controlled ablations on context scaling and agent co-training that were previously gated behind proprietary runs.