A new arXiv preprint by Simon et al. argues a scientific theory of deep learning is emerging, termed "learning mechanics," that characterizes training dynamics, representations, weights, and performance via five research strands: solvable idealized settings, tractable limits, macroscopic laws, hyperparameter theories, and universal behaviors (Simon et al., 2026, https://arxiv.org/abs/2604.21691).

This builds on and synthesizes prior results including scaling laws for neural language models that quantify performance predictability with model size, data, and compute (Kaplan et al., 2020, https://arxiv.org/abs/2001.08361) and the neural tangent kernel limit that provides exact dynamics for wide networks in the infinite-width regime (Jacot et al., 2018, https://arxiv.org/abs/1806.07572); original source coverage underemphasized how these converge with mechanistic interpretability to yield falsifiable, coarse-grained predictions, a gap this synthesis addresses by highlighting their shared focus on dynamics over static analysis.

The collected evidence indicates a rigorous theory is imminent, transforming AI research from empirical scaling experiments into a predictive, mechanics-based science and likely reprioritizing efforts toward deriving quantitative laws for realistic systems rather than solely pursuing larger models for the next decade; common objections that fundamental theory is impossible are refuted by the growing body of tractable limits and universal behaviors already matching empirical observations across architectures.