LLMs Advance Schema-Adaptive Clinical Reasoning for Generalizable Multimodal Diagnostics
arXiv paper demonstrates LLM-based tabular embeddings enable zero-shot schema transfer and SOTA multimodal dementia diagnosis outperforming neurologists, synthesizing Med-PaLM and Med-Gemini results to highlight generalizable healthcare AI progress.
Researchers introduced Schema-Adaptive Tabular Representation Learning, converting EHR variables into natural language statements encoded by pretrained LLMs to generate transferable embeddings without retraining or feature engineering (https://arxiv.org/abs/2604.11835). The method integrates with MRI data for dementia diagnosis, reporting state-of-the-art results on NACC and ADNI cohorts and successful zero-shot transfer to unseen schemas, exceeding clinical baselines and board-certified neurologists in retrospective tasks.
This builds on Med-PaLM's demonstration of LLMs encoding clinical knowledge (https://arxiv.org/abs/2212.13138) and Med-Gemini's multimodal medical capabilities (https://arxiv.org/abs/2404.18419), addressing the schema heterogeneity in real-world EHRs that prior tabular ML and early multimodal models consistently failed to generalize across. Original source coverage understates the dataset-shift patterns documented in deployment failures of COVID-era diagnostic tools and narrow EHR models, which this semantic LLM encoding directly mitigates by leveraging pretrained language understanding rather than statistical feature alignment.
Viewed through advancing schema-adaptive clinical reasoning, the work reveals a pathway to foundation models that synthesize structured records with imaging beyond siloed training sets, potentially improving diagnostic equity in heterogeneous health systems; however, the retrospective design leaves prospective validation and hallucination risks unaddressed per the synthesized sources.
AXIOM: LLMs can now turn varied medical record formats into semantic embeddings that transfer zero-shot, enabling diagnostic tools combining tables and scans to work across hospitals and outperform specialists on dementia tasks.
Sources (3)
- [1]Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning(https://arxiv.org/abs/2604.11835)
- [2]Large Language Models Encode Clinical Knowledge(https://arxiv.org/abs/2212.13138)
- [3]Med-Gemini: Capabilities of Gemini Models in Medicine(https://arxiv.org/abs/2404.18419)