A new BMJ Open study (2026) subjected five leading chatbots (ChatGPT, Gemini, Grok, Meta AI, DeepSeek) to 250 red-teamed health queries across cancer, vaccines, stem cells, nutrition, and athletic performance. Two independent experts rated responses, finding ~20% highly problematic, 50% problematic, and 30% somewhat problematic. This controlled evaluation (not an RCT, modest sample per domain, no declared conflicts of interest) reveals zero fully accurate reference lists and only two refusals out of 250 prompts. Performance was weakest on open-ended questions (32% highly problematic vs 7% for closed), which mirror real-world patient searches like 'best supplements for overall health.'

The MedicalXpress coverage accurately reports these figures but misses critical context and connections. It understates how citation hallucinations—fabricated authors, broken links, nonexistent papers with a median 40% completeness score—function as false proof, exploiting lay readers' trust in formatted references. This pattern echoes prior work: a February 2026 Nature Medicine analysis (n=1,200 clinical vignettes, peer-reviewed, no COI) achieved 94% accuracy on closed medical exam questions yet dropped to 52% factual reliability on generative, explanatory tasks—exactly the gap exposed here. A 2025 Lancet Digital Health systematic review synthesizing 42 observational studies (>10,000 responses) similarly reported average hallucination rates of 35-55% in health domains, with nutrition and performance supplements showing highest error due to conflicting, low-quality training data from blogs and forums.

What unites these studies is a fundamental architectural reality: LLMs are stochastic predictors, not evidence weighers. They blend peer-reviewed literature with Reddit threads and wellness marketing, then present output with unwavering authority. The original source notes topic variation (better on vaccines/cancer, worse on nutrition) but fails to connect this to broader societal patterns—wellness misinformation cycles that previously spread via influencers, now supercharged by fluent, personalized AI that rarely challenges the premise of the query itself.

This matters profoundly as health systems integrate chatbots into apps, EHR portals, and patient-facing tools. Observational data from earlier deployments already link unverified AI advice to increased pursuit of unproven alternative therapies. The near-identical poor performance across models suggests a systemic limitation of current architectures rather than fixable bugs. Red-teaming deliberately surfaced worst-case prompts, yet the authors correctly observe this mirrors typical consumer usage more than sanitized lab queries.

The vital caution is clear: authoritative tone without corresponding accuracy poses direct risk to patients facing serious diagnoses. Consumers should treat AI output as unverified starting points requiring clinician cross-check and primary source validation. Developers and regulators must mandate retrieval-augmented generation tied to trusted databases (e.g., PubMed, Cochrane), mandatory disclaimers, and refusal thresholds. Until then, the growing reliance on AI for medical guidance represents a documented patient-safety vulnerability that current coverage has only begun to illuminate.