Children Acquire Language From 10-100x Fewer Tokens Than LLMs

The data efficiency gap arises because children extract structured representations from sparse, noisy input without gradient descent over internet-scale corpora. Developmental corpora such as CHILDES record average annual exposure below 10 million words yet yield robust generalization on novel sentence structures. In contrast, the BabyLM 2023 challenge showed models trained on 100 million tokens still lag behind 5-year-olds on BLiMP and GLUE subsets by 15-25 points.

Reverse-engineering child learning requires inductive biases beyond next-token prediction. Lake et al. 2017 demonstrated that Bayesian program induction over compositional primitives matches human few-shot performance on concept learning tasks where transformers fail. Recent hybrid neuro-symbolic models incorporating core knowledge priors reduce required tokens by an order of magnitude on ARC and MiniSCAN while preserving accuracy.

Operationally this implies training pipelines must shift from raw scale to curriculum design and active sampling. Companies deploying retrieval-augmented systems already report 3-5x data reduction when priors match domain structure. Sustained gains will require new benchmarks that penalize token count rather than reward it.

Next steps include controlled longitudinal studies tracking token-equivalent exposure against model checkpoints and regulatory pressure on compute disclosure to surface efficiency metrics.