AI Chatbots Give Accurate Vaccine Facts That Research Links to Weaker Intentions

The evaluation tested three leading models on common parental questions about childhood vaccines. All outputs scored high on factual retrieval yet repeatedly used descriptive language—such as labeling a risky behavior “common” or framing vaccination solely as an altruistic act—that message-effects experiments have shown can reduce uptake or backfire. These patterns echo documented failures of accurate public-health campaigns on binge drinking and smoking that produced no change or opposite behavioral shifts.

Existing regulatory drafts, including the Anthropic-circulated bill, set thresholds only for models that could “imminently kill” more than 50 people. They omit requirements for pre-deployment behavioral testing of health messages, even though modifiable risk factors account for nearly half of U.S. cancer deaths. Lawsuits filed by states against OpenAI similarly target overt errors rather than accurate phrasing with measurable adverse effects.

Next steps include embedding randomized message trials inside model-release pipelines and mandating publication of behavioral outcome data alongside accuracy benchmarks. Without such tests, scaled deployment of autonomous patient-communication tools risks amplifying the very declines in vaccination and screening rates now observed nationally.