Type Theory and Neural Networks: Paths to Intrinsically Typed AI Code Generators
Current LLM type handling via post-training fixes is inefficient; native integration during training via neurosymbolic and categorical methods remains undercovered but evidenced in formal methods research.
LLMs predict tokens without built-in type awareness, relying on post-hoc parsing and validation for dependently typed languages such as Idris, Lean, and Agda (Gavranovic, 2026). The source identifies deficiencies in low-granularity error feedback loops and pre-token constrained decoding that mask invalid choices based on partial types.
Original coverage overlooks how training objectives could incorporate type inhabitation directly, as seen in related neurosymbolic approaches (Chaudhuri et al., 'Neurosymbolic Programming,' 2021). Patterns from theorem-proving LLMs like those integrated with Lean show improved performance when type information guides search (Polu et al., 'Formal Mathematics Statement Curriculum Learning,' 2022).
This intersection suggests that category-theoretic models of both types and networks could enable end-to-end differentiable typed generation, addressing the fixed 'List Token' output limitation and allowing models to internalize type constraints during training (Gavranovic, 2026; Polu et al., 2022).
AXIOM: Training models to output inherently typed structures instead of token lists could cut invalid code rates and enable reliable formal verification in AI systems.
Sources (3)
- [1]Types and Neural Networks(https://www.brunogavranovic.com/posts/2026-04-20-types-and-neural-networks.html)
- [2]Formal Mathematics Statement Curriculum Learning(https://arxiv.org/abs/2202.01344)
- [3]Neurosymbolic Programming(https://arxiv.org/abs/2107.00677)