arXiv:2611.06995: AI agents in Lean favor marked gaps over dense centers across 4.7M definitions
arXiv:2611.06995 quantifies systematic preference divergence between autonomous agents and human formalizers on Lean proof tasks. Agents over-attend to marked gaps and surprising bodies while avoiding dense semantic centers. The findings supply concrete metrics for retraining agent policies in interactive theorem proving.
The study introduces simulated collaboration to compare agent and human step selection on mathlib-scale repositories containing 4.7 million definitions and over 6,000 commits. Agents exhibited stronger attraction to explicitly marked sorry gaps, avoidance of high-degree nodes in the proof graph, and preference for surprising local bodies that they then regularized toward common patterns. These patterns were quantified by tracking attention to surface syntax versus semantic connectivity at each tactic step.
Human commit histories reveal consistent prioritization of densely connected lemmas that reduce global search space, while agents under-sampled those regions even when they were available. The divergence was largest on goals containing nested inductive structures and non-obvious type-class resolutions. The methodology isolates preference from capability by replaying identical proof states for both agents and recorded human traces.
Operational implication is that current agent loops will systematically under-contribute to the connective tissue of large formal libraries unless explicit re-ranking or human-in-the-loop signals are added. Future training pipelines can use the reported divergence metrics as auxiliary losses to align step selection distributions with human commit patterns observed in the same repositories.
No public agent release or Lean tactic library update has incorporated these alignment signals as of the paper date.
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2611.06995)
- [2]Supporting Source(https://leanprover-community.github.io/mathlib_docs/)