Multi-Token Prediction Induces Backward Planning Circuits in Transformers
MTP fosters reverse reasoning via gradient decoupling, connecting mechanistic interpretability to the development of planning in agentic AI systems.
Huang et al. (arXiv:2604.11912) demonstrate MTP outperforms NTP on graph path-finding, Countdown, and boolean SAT tasks. In a two-layer transformer on star graphs, MTP creates a two-stage reverse reasoning process: first attending to the end node then tracing intermediate nodes backward, arising from gradient decoupling that supplies cleaner training signals than NTP.
Original coverage of the paper overlooks explicit ties to emergent reasoning patterns documented in Wei et al. (arXiv:2201.11903), where chain-of-thought prompting elicits multi-step planning, and to Anthropic's mechanistic interpretability work (Towards Monosemanticity, 2023) on dictionary learning that decomposes similar circuits. NTP's local focus often fails to capture global structure; MTP's objective inherently biases optimization toward interpretable, goal-directed circuits that mirror test-time planning in models such as OpenAI o1.
These mechanistic insights reveal MTP as a training-time analogue to inference-time search, illuminating the broader shift toward agentic AI. By decoupling gradients across tokens, transformers develop robust planning capabilities without explicit supervision, addressing limitations in prior NTP-only systems and offering a pathway to verifiable reasoning circuits that can be inspected and engineered.
AXIOM: Transformers trained with multi-token prediction learn to first attend to the goal token then backtrack through intermediate states, an emergent reverse-planning mechanism that supplies the mechanistic foundation for more interpretable and reliable agentic AI.
Sources (3)
- [1]How Transformers Learn to Plan via Multi-Token Prediction(https://arxiv.org/abs/2604.11912)
- [2]Chain-of-Thought Prompting Elicits Reasoning in Large Language Models(https://arxiv.org/abs/2201.11903)
- [3]Towards Monosemanticity: Decomposing Language Models With Dictionary Learning(https://www.anthropic.com/research/towards-monosemanticity)