Large language models exhibit systematic deficiencies in mechanistic understanding required for biomolecular modeling across scales.

The BioMol-LLM-Bench framework (arXiv:2604.03361) assesses 13 models on 26 tasks at four difficulty levels integrating computational tools, finding strong classification performance but persistent weakness on regression tasks critical for molecular property prediction in drug discovery (Xu et al., 2026). This aligns with AlphaFold's specialized success in structure prediction while exposing gaps in general LLMs for quantitative multi-scale problems (Jumper et al., Nature, 2021).

Chain-of-thought data yields limited gains and can degrade biological task results; hybrid mamba-attention architectures handle long sequences more effectively than standard transformers; supervised fine-tuning improves specialization yet harms generalization, patterns also documented in Galactica's scientific domain evaluations (Taylor et al., arXiv:2211.09085).

Original abstract and related coverage overlooked explicit connections to prior LLM-for-science failures at causal reasoning, understating how these cross-scale results counter narratives of imminent LLM-driven breakthroughs in pharmaceutical development by highlighting irreducible gaps versus mechanistic simulation methods.