Context Windows of 128K Tokens Allow LLMs to Retain Full Proof States Where Human Working Memory Caps at 7 Chunks

The provided analysis correctly identifies working-memory expansion as the primary mechanism behind recent LLM math gains. Alloway and Alloway 2010 tracked 5-year-olds and found working-memory scores predicted numeracy six years later after controlling for IQ, confirming the independent constraint. Current models such as GPT-4o and Claude 3.5 maintain 128K-token contexts that hold hundreds of equations simultaneously, removing the chunking bottleneck humans face even after expertise acquisition.

Yet the same memory advantage exposes deeper limits. Models still require explicit external search or verifier loops on IMO shortlist problems and fail to generate novel conjectures outside training distributions. AlphaGeometry achieved 25 of 30 geometry problems only after hybrid symbolic engines supplied the missing deductive steps, showing context size alone does not produce autonomous innovation.

Operationally this means deployment pipelines must retain human oversight for open-ended research tasks. Pure scaling of context length will plateau once problems exceed available tokens or require abstraction not present in the training corpus. Verification systems and formal libraries remain necessary scaffolds rather than temporary crutches.

Future systems will therefore combine larger contexts with external memory stores and formal checkers, but will continue to underperform on problems demanding original conceptual compression beyond statistical recombination.