Pennsylvania Governor Pushes for Stricter AI Chatbot Regulations

Pennsylvania Governor Josh Shapiro is urging state agencies to create stricter regulations for artificial intelligence chatbots, which he argues pose risks of misleading information and emotional harm, particularly to children. This initiative could position Pennsylvania alongside other states actively seeking to impose regulations as young users increasingly turn to chatbots like ChatGPT and Meta AI. During his budget address last week, Shapiro emphasized the urgency of the situation: “This space is evolving rapidly. We need to act quickly to protect our kids.”

A survey conducted last year by the nonprofit Common Sense Media found that a majority of U.S. teenagers use chatbots, with one in three utilizing them for social interactions and relationships. These teens reported employing chatbots for various purposes, including conversation practice, emotional support, role-playing, and even forming friendships and romantic connections. Shapiro highlighted that without regulation, vulnerable users, particularly children, could face significant emotional risks. “Some kids are just too young to understand the difference between AI and a real person,” he stated.

Significant concerns have been raised regarding the impact of AI interactions on mental health. In January 2023, Google settled multiple lawsuits alleging that its Character.AI contributed to mental health crises, including a case involving a Florida mother whose son died by suicide after developing a relationship with a chatbot. In response, Shapiro proposed several measures, including the implementation of age verification, parental consent, and a ban on chatbots generating sexually explicit or violent content involving minors. He also advocates for directing users who mention self-harm or violence to appropriate resources and reminding users that they are not interacting with a human.

Despite the pressing need for regulation, questions remain regarding the feasibility of enforcing these new requirements. Hoda Heidari, a professor in ethics and computational technologies at Carnegie Mellon University, noted, “The devil is in the details.” She expressed skepticism about the attainability of these goals while acknowledging the importance of addressing the broader aims of regulation.

Age verification has gained traction among regulators as a potential measure to protect children from inappropriate content. However, Heidari pointed out that similar challenges could arise in the context of chatbots. “These questions are very challenging both from a technical perspective of how you verify the age and then from a legal perspective of what to do with the resulting data,” she explained. The ease with which individuals can falsify identification online complicates this issue further. For instance, many websites use age gates requiring users to input their birth date, but users can easily bypass this by entering any date.

Ensuring that chatbots do not produce violent or sexually explicit content involving minors presents another significant challenge. Heidari noted that while AI companies are actively exploring methods to block the generation of child sexual abuse material, existing safeguards can be circumvented. “Think of all the ways in which you can prompt a chatbot to generate the same kind of content you have in mind,” she cautioned. “Ensuring that you never see a kind of behavior is actually quite challenging.”

Shapiro has also called on lawmakers to develop legislation aimed at protecting children and other vulnerable users from the risks associated with chatbot use. A bipartisan bill currently under consideration in the state Senate would establish “age-appropriate standards” and implement safeguards against content that encourages self-harm, suicide, or violence. This legislation would also require chatbots to direct users to self-harm crisis resources when high-risk language is detected. Nevertheless, enforcement and penalties for non-compliance remain uncertain. “These are the kinds of requirements that are going to be very hard to enforce,” Heidari stated, emphasizing the importance of agencies continuing their efforts despite these challenges.

The landscape of AI regulation in the United States has been characterized by rapid, unregulated growth reminiscent of the Wild West. With the lack of a cohesive federal framework, it remains unclear how the regulatory environment will evolve under the second Trump Administration. In December, President Trump signed an executive order discouraging state-level regulation and pledging to challenge “onerous and excessive laws” that may hinder innovation. This order also promised a “minimally burdensome national policy framework for AI” and established an AI litigation task force to address inconsistencies with state laws.

As state governments pursue their own regulatory measures, California has emerged as a frontrunner, having enacted a range of legislation since 2024 focused on improving transparency, safety, and accountability in AI models. New York has adopted similar initiatives. As Pennsylvania and other states develop their regulatory frameworks, Heidari warned of creating a “patchwork system” that could complicate compliance for AI companies. “This doesn’t bode well for a unified, coherent system where companies and other AI stakeholders can decide what to do,” she remarked.

Without a federal AI policy in place, it is likely that populous states like California and New York will set the regulatory tone. Heidari noted that companies are unlikely to implement different chatbot platforms across various states, preferring to adhere to regulations established by larger states. “They get to set the tone for the whole country,” she explained, suggesting that it would be impractical for AI companies to navigate varying laws in different jurisdictions.

Shapiro’s renewed efforts could position Pennsylvania as a significant player in the ongoing dialogue surrounding AI regulation in the U.S. Heidari praised the Shapiro administration’s responsible approach and its commitment to engaging with stakeholders and experts. “Otherwise, there is a way in which regulation can just be paying lip service to certain political agendas without having any impact,” she cautioned. The challenge now lies in translating these discussions into effective, enforceable regulations that genuinely protect users, particularly children, in a rapidly evolving digital landscape.