California lawmakers are pushing to ban AI therapists as interest in chatbot-based mental healthcare has surged in recent years.
Under Senate Bill 903, companies would be banned from advertising chatbots as therapy, while AI systems would not be allowed to make therapeutic decisions without review by a licensed professional.
The bill would also require patients to give permission before healthcare providers use AI to record therapy sessions or triage care, which means assessing patients to determine the level of treatment they need.
State senator Steve Padilla of San Diego, who introduced the legislation, said earlier this year: “AI has many upsides, and the possibility to improve the lives of Californians across the state, but only if we ensure it is used responsibly.”
“AI algorithms are not fit to take over the job of human therapists, who have skills and training that AI is incapable of replicating,” he added.
“We must act and place guardrails that prevent AI algorithms from being deployed in a way that is potentially harmful to patients.”
Critics of the bill say it could restrict legitimate uses of AI and make it harder for healthcare providers to use AI tools to direct patients towards treatment.
Robert Boykin, California executive director of industry group TechNet, told CalMatters: “At a time when every county in California faces a shortage of behavioural healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster.”
The National Union of Healthcare Workers, which supports the bill, has filed a complaint with California regulators alleging that the Kaiser Permanente health system used an automated e-visit system without clinician oversight to make care recommendations and provide initial assessments of patients who may have anxiety or depression.
Kaiser Permanente said it “does not use AI to diagnose patients, make clinical decisions, or determine medical necessity,” and has argued that the union mischaracterised its process.
Data indicates strong interest in using AI for therapy, although health experts warn that chatbot-based care needs further study and regulation before it can be compared with conventional medical care.
A 2025 study from RAND, Brown University School of Public Health and Harvard University found that around one in eight adolescents and young adults were using chatbots for mental healthcare.
Jonathan Cantor, a senior policy researcher at RAND, told the Brown University website: “There are few standardised benchmarks for evaluating mental health advice offered by AI chatbots, and there is limited transparency about the datasets that are used to train these large language models.”
A study this year from the American Psychological Association found that 77 per cent of psychologists had spoken with patients who had used AI for support, engagement or other reasons.
In October, OpenAI, the maker of ChatGPT, disclosed that an estimated 1.2m users a week sent messages to the chatbot containing “explicit indicators of potential suicidal planning or intent.”
Health experts say the growing use of AI-based support has come amid difficulties accessing in-person mental healthcare.
Hamilton Morrin, a clinical fellow in neuropsychiatry at King’s College London, told Harvard Business Review in June: “Given long waiting lists and difficulty accessing mental health care and therapy in many countries, it’s perhaps not a surprise that increasing numbers [of people] are turning to generative AI for support with their emotional wellbeing.”
The makers of popular AI applications, which can hold lifelike conversations with users, have faced lawsuits alleging that chatbot interactions played a role in suicides among young people.
