arXiv 2609.21096: Forman-Ricci curvature on attention graphs flags hallucination via self-attention over-reliance

The method constructs attention graphs per head, computes Forman-Ricci curvature to locate bottlenecks, and extracts semi-local plus global flow statistics in a single forward pass. Evaluation covered multiple LLMs on two standard hallucination benchmarks, yielding consistent gains over attention-only and multi-sample baselines while remaining architecture-agnostic. Hallucinated generations repeatedly displayed three signatures: self-loop dominance, diffused early-token retrieval, and final-layer over-squashing.

These topological markers align with earlier observations in graph-neural-network literature that negative curvature regions impede message passing; the 2026 paper supplies the first direct mapping from such regions to token-level hallucination in causal LLMs. Prior detection work focused on logit entropy or consistency across samples missed the structural bottleneck signal captured here. The single-pass constraint also removes the latency penalty of methods requiring multiple generations.

Operational impact appears in inference pipelines that can now attach a lightweight curvature monitor to the final layer without retraining. Deployment records from production RAG systems indicate that routing low-curvature generations to verification stages reduces downstream factual errors by measurable margins on retrieval-augmented tasks.

Next steps include extending curvature tracking to intermediate layers and testing whether targeted attention regularization during fine-tuning reduces the identified signatures on held-out benchmarks.