Deakin Position Paper Flags Regulatory Gaps in AI Mental Health Tools

The paper maps three AI touchpoints—patient-facing chat tools, clinician decision support, and system-level risk flagging—against current evidence. It notes that passive smartphone data and session summarization can reduce administrative load but also risk inaccurate risk detection or altered disclosure patterns. Observational data cited show early uptake in high-income settings, yet no large RCTs confirm net clinical benefit or harm reduction. Analysis reveals the coverage underplays liability allocation when an AI flags suicide risk incorrectly; analogous FDA device recalls for cardiac algorithms show miscalibration can persist for years without mandated post-market audits. Privacy erosion from continuous mood tracking also receives limited attention despite GDPR enforcement actions against health apps.

Rapid post-2022 generative AI diffusion has outpaced mental health governance, mirroring the 2010s opioid prescribing software failures where unvalidated algorithms amplified over-treatment. Bias in training corpora drawn from English-language therapy transcripts risks poorer performance for non-Western presentations of distress. The paper correctly identifies escalation pathways as essential but stops short of specifying minimum dataset diversity thresholds required for regulatory approval.

Next steps include embedding independent safety testing before deployment and creating clinician-AI liability statutes. International bodies such as WHO and EMA are positioned to issue joint guidance within two years if member states prioritize the issue.