States Struggle with Regulation of AI Mental Health Apps

As the demand for mental health support increases, particularly during challenging times, the rise of AI mental health applications has prompted state regulators in the United States to take action. Several states have passed laws in 2023 to regulate AI-driven “therapy” apps, yet these measures struggle to keep pace with the rapidly evolving technology landscape. Mental health advocates and app developers argue that the current framework does not adequately protect users or hold app creators accountable for potential harm.

According to Karin Andrea Stephan, CEO and co-founder of the mental health chatbot app Earkick, “The reality is millions of people are using these tools and they’re not going back.” This sentiment highlights the urgency of addressing the regulatory gaps surrounding AI applications in mental health.

State-Level Regulations Fall Short

The regulatory landscape varies significantly across states. For instance, Illinois and Nevada have enacted outright bans on AI applications that claim to provide mental health treatment, imposing fines of up to $10,000 in Illinois and $15,000 in Nevada for violations. Conversely, Utah has introduced limitations on therapy chatbots, mandating that they safeguard users’ health information and clearly disclose that they are not operated by humans.

States like Pennsylvania, New Jersey, and California are also exploring regulatory measures. However, many of these laws do not apply to generic chatbots, such as ChatGPT, which are frequently utilized for mental health advice despite not being marketed as therapeutic tools. Serious incidents have occurred where users, after interacting with these chatbots, experienced severe mental health crises, leading to lawsuits.

Vaile Wright, who oversees healthcare innovation at the American Psychological Association, acknowledges the necessity of these apps in light of a nationwide shortage of mental health providers, high costs of care, and unequal access for patients with insurance. She emphasizes the potential of scientifically grounded mental health chatbots that are created with expert input and are monitored by professionals. “This could be something that helps people before they get to crisis,” Wright stated.

Federal Oversight and the Need for Comprehensive Regulation

In early November 2023, the Federal Trade Commission announced it would investigate seven AI chatbot companies, including those associated with major tech platforms like Google and Facebook, to assess how they evaluate and mitigate the potential negative impacts of their technologies on children and adolescents. Additionally, the Food and Drug Administration plans to convene an advisory committee to review generative AI-enabled mental health devices.

Wright suggests that federal agencies might consider imposing marketing restrictions on chatbots, limiting addictive practices, and mandating that companies disclose they are not medical providers. She also advocates for legal protections for individuals who report unethical practices by these companies.

Nonetheless, the landscape of AI applications in mental health is complex and diverse. From “companion apps” to “AI therapists,” the varying purposes of these applications complicate regulatory efforts. Some states are targeting companion apps designed solely for companionship without venturing into mental health care.

While some apps have opted to restrict access in response to state regulations, others, like Earkick, continue to grapple with compliance. Stephan noted that the legal framework remains ambiguous, particularly in Illinois, where the company has not limited access. Originally, Earkick avoided referring to its chatbot as a therapist but later adopted the term based on user feedback. Currently, the chatbot is promoted as a “chatbot for self-care,” with the caveat that it does not provide diagnoses.

Stephan expressed concern about the regulators’ ability to keep up with the rapid advancements in AI technology, stating, “The speed at which everything is evolving is massive.”

The need for careful, evidence-based approaches to AI in mental health is underscored by ongoing research. In March 2023, a team from Dartmouth University published a randomized clinical trial on a generative AI chatbot called Therabot, designed to assist individuals diagnosed with anxiety, depression, or eating disorders. The study indicated that users rated Therabot similarly to human therapists, with participants reporting significantly reduced symptoms after eight weeks of use.

Nicholas Jacobson, a clinical psychologist involved in the research, noted the necessity for larger studies to validate Therabot’s effectiveness across broader populations. He cautioned against hasty regulations that could inhibit developers who prioritize safety and efficacy.

As discussions around AI mental health applications continue to evolve, stakeholders, including regulators and mental health advocates, are open to revisiting existing laws. Yet, Kyle Hillman, who lobbied for the Illinois and Nevada bills through the National Association of Social Workers, highlighted a critical point: “Not everybody who’s feeling sad needs a therapist.” For those experiencing genuine mental health crises, he argued that providing a chatbot as an alternative to professional help is an inadequate solution.

The ongoing dialogue surrounding AI mental health applications reflects the broader challenges of integrating new technologies into healthcare while ensuring user safety and accountability. The balance between innovation and regulation remains delicate, necessitating ongoing collaboration among developers, regulators, and mental health professionals.