BioEVAL benchmark records 90% MCQ accuracy on 359 retained items across 11 bioengineering subfields

BioEVAL assembled 380 MCQs, 218 literature synthesis tasks, and 10 multimodal items spanning 11 bioengineering subfields. After centralized review and blinded cross-group audit, 21 MCQs were removed, leaving 359 for scoring. Cloud-scale models including ChatGPT, Gemini, and Grok plus local GPU-deployable models were tested under standardized protocols.

Highest reported scores reached 90% on MCQs, 0.72 similarity on literature synthesis, and 80% on the multimodal subset. Performance varied sharply by subfield, exposing gaps in experimental reasoning versus factual recall. The audit process flagged items where model errors clustered, indicating systematic weaknesses in frontier bioengineering tasks.

Existing biomedical benchmarks emphasize recall; BioEVAL shifts focus to PhD-level experimental reasoning and image interpretation. Multi-institutional authorship and extensible contribution protocol reduce single-group bias while enabling ongoing item addition. Operational deployment of top models remains limited by subfield variance and small multimodal sample size.

Next evaluation round will expand multimodal items and enforce version-controlled model submissions to track incremental gains against the 90% MCQ ceiling.