Spectral Phase Transitions Distinguish Reasoning in 11 LLMs Across Architectures
Transformers exhibit universal spectral compression and reversal on reasoning tasks, enabling perfect preemptive correctness prediction via activation geometry.
Lede: Spectral geometry in hidden activations separates reasoning from factual recall via phase transitions and alpha compression in 9 of 11 models spanning Qwen, Pythia, Phi, Llama and DeepSeek-R1 families (arXiv:2604.15350).
Liu et al. report reasoning spectral compression with lower alpha values (p < 0.05), instruction tuning spectral reversal, architecture-dependent prompt-response regimes, and spectral scaling law alpha_reasoning proportional to -0.074 ln N in Qwen base models (R^2 = 0.46) (arXiv:2604.15350). Original coverage omitted explicit ties to prior phase transition work in Power et al. on grokking (arXiv:2201.02177) where training dynamics exhibit sudden shifts paralleling the observed spectral punctuation at reasoning boundaries.
Token-level spectral cascade shows per-token alpha synchronization decaying exponentially across layers and weaker for reasoning than factual tasks (arXiv:2604.15350). This connects to induction head mechanisms in Olsson et al. (arXiv:2202.07785) but adds geometric compression absent from that circuit-level account. Spectral alpha predicts correctness with AUC=1.000 in Qwen2.5-7B late layers and mean AUC=0.893 across six models prior to answer generation (arXiv:2604.15350).
AXIOM: Spectral alpha in transformer hidden states flips direction after instruction tuning and reaches AUC 1.0 for predicting correct reasoning outputs before generation.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2604.15350)
- [2]Grokking: Generalization Beyond Overfitting(https://arxiv.org/abs/2201.02177)
- [3]In-context Learning and Induction Heads(https://arxiv.org/abs/2202.07785)