Pennsylvania sues Character.AI, alleging chatbots posed as licensed medical professionals

Pennsylvania has sued Character Technologies Inc., the company behind Character.AI, seeking to stop chatbots on the platform from posing as licensed medical professionals, Governor Josh Shapiro said.

Shapiro said the case is the first lawsuit of its kind brought by a U.S. governor over alleged chatbot impersonation of medical practitioners.

Allegations in the complaint In a complaint filed in the Commonwealth Court of Pennsylvania, the state alleges that Character.AI hosts user-created chatbots that claim to practice medicine.

Pennsylvania said an investigation by its AI task force, created in February, identified a chatbot named “Emilie” that told an investigator posing as a patient that it was licensed to practice psychiatry in Pennsylvania and the United Kingdom and provided a fabricated license number. The chatbot also indicated it could prescribe medication, the state said.

What the state is seeking Pennsylvania is seeking an injunction to prevent Silicon Valley-based Character.AI from violating Pennsylvania laws that prohibit the unauthorized practice of medicine.

Character.AI response and prior litigation Character.AI said user-created characters on its site are fictional and intended for entertainment and role playing, and that it prioritizes user safety.

Pennsylvania also cited other legal challenges involving the company, including a Kentucky lawsuit filed in January related to child safety and a wrongful-death settlement in Florida tied to allegations about a chatbot’s influence on a minor.