Perspective Paper in Nature Health Exposes Invisible Harms From Untraceable AI Chatbot Medical Advice

The paper centers on a documented case of a 60-year-old man who replaced table salt with sodium bromide after ChatGPT advice on chloride reduction, resulting in bromide toxicity, hallucinations and hospitalization. Because the interaction remained on the platform with no export or logging option, treating physicians could not reconstruct the causal pathway. The authors note this absence is structural rather than incidental, as current chatbot architectures prioritize user retention over auditability.

Related work in NEJM (2023) on large language models in medicine and a 2024 Lancet Digital Health analysis of AI-generated search summaries both identified hallucination rates above 20 percent on clinical queries yet stopped short of examining downstream traceability. The Nature Health perspective connects these gaps to passive exposure via social media feeds and search engines, where users encounter unvetted content without initiating a query.

Proposals include mandatory user-accessible conversation exports, standardized adverse-event reporting channels to AI firms, and pre-vetting requirements for health summaries in search and social platforms. Implementation would require regulatory extension of existing medical device oversight frameworks to generative systems, a step not yet taken by FDA or EMA.

Next steps hinge on whether major platforms voluntarily adopt logging or face legislative mandates; absent either, observational studies of chatbot-attributable harm will remain underpowered.