As interest in chatbot-based mental health services surges, California legislators are taking decisive action to regulate the use of artificial intelligence in therapeutic settings. A recent study shows that young people are increasingly turning to AI for emotional support, prompting concerns among officials regarding the safety and efficacy of these digital solutions.
Legislative Measures to Regulate AI in Therapy
Senate Bill 903 aims to prohibit companies from marketing chatbots as substitutes for human therapists. The proposed legislation stipulates that AI systems must not make therapeutic decisions independently; instead, any recommendations must be vetted by licensed professionals. Furthermore, the bill would necessitate patient consent before health providers can use AI technology to record therapy sessions or assess care needs.
State Senator Steve Padilla, who sponsored the bill, emphasised the importance of responsible AI usage in healthcare. “While AI holds great promise for enhancing the lives of Californians, we must ensure it is applied in a manner that prioritises patient safety,” Padilla stated. He argued that AI lacks the nuanced understanding and training that human therapists possess, making it unsuitable for assuming their roles.
Industry Concerns and Criticism
Despite the bill’s intentions, critics argue that it could hinder the effective use of AI tools in mental healthcare, especially in light of the existing shortage of behavioural health professionals across California. Robert Boykin, executive director of the industry group TechNet, expressed concern that the legislation might create unnecessary barriers to accessing timely mental health care. “With every county in California grappling with a lack of behavioural healthcare workers, SB 903 risks complicating the intake process and delaying patient care,” he remarked.
The National Union of Healthcare Workers, a supporter of the bill, recently filed a complaint against Kaiser Permanente, alleging that the health system utilised an automated “e-visit” system without proper clinician oversight, potentially compromising patient care. Kaiser Permanente has countered these claims, asserting that AI is not employed for diagnosing or making clinical decisions.
The Growing Use of AI in Mental Health
As mental health struggles continue to rise, especially among adolescents, there is a noteworthy increase in the number of individuals seeking support from AI chatbots. A 2025 study conducted by RAND Corporation, Brown University, and Harvard University found that approximately 12.5% of young people are using chatbots for mental health assistance.
Jonathan Cantor, a senior policy researcher at RAND, highlighted the need for standardised evaluation methods for AI-generated mental health advice. “The current lack of transparency regarding the datasets used to train these models raises significant concerns about the reliability of the support they provide,” he noted.
In a related survey conducted by the American Psychological Association, a staggering 77% of psychologists reported that they had interacted with patients who had used AI for various forms of support. This trend reflects a growing reliance on AI technologies in a landscape where access to in-person mental health services often remains limited.
The Implications of AI in Mental Health Care
The challenges of obtaining timely mental health care have led many to seek solace in generative AI solutions. Hamilton Morrin, a clinical fellow in neuropsychiatry at King’s College London, remarked on the phenomenon, stating, “Given the long waiting times for mental health services, it is not surprising that people are turning to AI for support with their emotional well-being.”
However, the rise of AI in this sensitive field is not without its controversies. Developers of various AI applications have faced lawsuits alleging that their tools may have contributed to tragic outcomes, including youth suicides, underscoring the need for stringent regulation and oversight.
Why it Matters
The conversation surrounding AI in mental health care is of paramount importance as it intersects with the well-being of countless individuals. By implementing regulations like Senate Bill 903, California lawmakers aim to balance the innovative potential of AI with the imperative of patient safety. As more individuals turn to technology for emotional support, understanding the limitations and risks of AI chatbots is crucial to ensuring that these tools complement, rather than replace, human care. The ongoing dialogue about AI’s role in mental health is not just a legal issue; it is a matter of life and health that will shape the future of therapeutic practices.
