As the use of AI-driven chatbots for mental health support surges, California lawmakers are taking decisive action to regulate their implementation. A new legislative proposal aims to prevent companies from marketing these chatbots as substitutes for traditional therapy, reflecting growing concerns about the implications of AI in sensitive healthcare settings.
Legislative Overview
Under the proposed Senate Bill 903, companies would be prohibited from advertising AI chatbots as therapeutic tools. The legislation mandates that any therapeutic decisions made by AI systems must undergo review by licensed mental health professionals. Additionally, the bill will require explicit consent from patients before health providers can utilise AI for recording therapy sessions or triaging care.
State Senator Steve Padilla, representing San Diego and the bill’s author, emphasised the potential benefits of AI, stating, “AI has many upsides, and the possibility to improve the lives of Californians across the state, but only if we ensure it is used responsibly.” However, he firmly believes that algorithms cannot replace the nuanced skills of trained therapists, advocating for measures to safeguard patients from potentially harmful AI practices.
Concerns from the Industry
Critics of SB 903 argue that the proposed restrictions may hinder the effective deployment of AI in mental healthcare. Robert Boykin, executive director of TechNet in California, warned that the legislation could create bottlenecks in accessing care at a time when the state is grappling with a significant shortage of behavioural healthcare professionals. “At a time when every county in California faces a shortage of behavioural healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster,” he stated.
The National Union of Healthcare Workers, which supports the bill, recently filed a complaint against Kaiser Permanente, alleging that their automated “e-visit” system operated without proper oversight from clinicians. Kaiser Permanente has countered that it does not use AI for diagnostic purposes, asserting that the union has misrepresented its processes.
Increasing Demand for AI Therapy
Recent studies indicate a rising trend among younger demographics turning to AI chatbots for mental health support. Research from 2025 by RAND, the Brown University School of Public Health, and Harvard University found that approximately one in eight adolescents and young adults are seeking assistance from these digital platforms.
Jonathan Cantor, a senior policy researcher at RAND, highlighted the challenges in assessing the reliability of AI-generated mental health advice. “There are few standardized benchmarks for evaluating mental health advice offered by AI chatbots, and there is limited transparency about the datasets that are used to train these large language models,” he noted.
Furthermore, a 2023 study conducted by the American Psychological Association revealed that 77% of psychologists have interacted with patients who have sought support through AI channels, indicating a significant intersection between traditional therapy and emerging technologies.
The Risks of AI in Mental Health
Despite the growing interest in chatbot-based mental healthcare, experts caution against unregulated use. In October, OpenAI disclosed that its popular chatbot, ChatGPT, receives an estimated 1.2 million messages weekly that contain “explicit indicators of potential suicidal planning or intent.” This alarming statistic underscores the urgent need for critical evaluation and oversight in AI applications related to mental health.
Hamilton Morrin, a clinical fellow in neuropsychiatry at King’s College London, pointed out that the appeal of generative AI as a mental health resource is partly due to difficulties in accessing in-person care. “Given long waiting lists and difficulty accessing mental health care and therapy in many countries, it’s perhaps not a surprise that increasing numbers [of people] are turning to generative AI for support with their emotional wellbeing,” he explained.
Why it Matters
The push for regulation of AI chatbots in mental healthcare highlights a pivotal moment in the intersection of technology and human well-being. As society grapples with the implications of integrating AI into sensitive areas such as mental health, California’s legislative efforts may serve as a crucial precedent. The balance between innovation and patient safety will be vital as the capabilities of AI continue to evolve, and ensuring that technology serves as a complement rather than a replacement for human professionals is essential to safeguarding mental health care’s integrity.
