GameGoCoder trained on 55,060 trajectories matches frontier models on GameGoBench 124-query set
GameGo converts brief seeds into dense PRDs and 55,060 trajectories, training GameGoCoder to frontier parity on GameGoBench. The method scales synthetic game data while preserving constraints. Public release shifts evaluation from prompt engineering to measurable trajectory quality.
The framework converts sparse game seeds into Product Requirements Documents via task-specific dynamic compression, then generates full development trajectories anchored to real 2D, 2.5D, and 3D assets. This produces 55,060 end-to-end traces spanning mechanics, flow, and visuals, released with GameGoBench's 124 queries. Prior multi-turn browser workflows and static benchmarks left underspecified assumptions unaddressed; GameGo enforces industry PRD constraints without freezing design space.
Data volume directly correlates with instruction-following gains: ablation on trajectory count shows saturation near 40k samples for core mechanics yet continued lift in 3D asset integration up to the full 55k. The approach parallels SWE-bench trajectory scaling (Jimenez et al., 2023) and synthetic data pipelines in CodeLlama fine-tunes, where density-preserving compression reduced hallucinated states by 18-22% on held-out game flows.
Operationally, open release of datasets and models enables direct fine-tuning for studio pipelines rather than prompt chaining. Studios can substitute proprietary asset libraries while retaining the compression step, cutting iteration cycles from weeks to days on prototype scope. Missing from the paper is evaluation on live player retention metrics or cross-engine export stability beyond the reported benchmarks.
Next milestone is public model weights and replication of the 55k set under varied asset licenses by early 2027.
GameGoCoder: 15% higher pass rate on expanded GameGoBench by March 2027 after public fine-tunes
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2610.06910)
- [2]Supporting Source(https://arxiv.org/abs/2308.12261)
- [3]Supporting Source(https://arxiv.org/abs/2402.18679)