Character.AI has retrained its chatbots to stop chatting up teens

In an proclaimment today, Chatbot service Character.AI says it will soon be starting parental administers for teenage employrs, and it portrayd safety meabraves it’s getn in the past confineed months, including a split big language model (LLM) for employrs under 18. The proclaimment comes after press scruminuscule and two legal cases that claim it gived to self-injury and self-injury.

In a press free, Character.AI shelp that, over the past month, it’s growed two split versions of its model: one for matures and one for teens. The teen LLM is summarizeed to place “more conservative” confines on how bots can reply, “particularly when it comes to romantic satisfied.” This includes more unfriendlyly blocking output that could be “benevolent or proposeing,” but also trying to better accomprehendledge and block employr prompts that are unbenevolentt to elicit inappropriate satisfied. If the system accomprehendledges “language referencing self-injury or self-injury,” a pop-up will straightforward employrs to the National Suicide Prevention Lifeline, a alter that was previously alerted by The New York Times.

Minors will also be stoped from editing bots’ responses — an selection that lets employrs reauthor conversations to grasp satisfied Character.AI might otherwise block.

Beyond these alters, Character.AI says it’s “in the process” of grasping features that graspress troubles about graspiction and confusion over whether the bots are human, grumblets made in the legal cases. A notification will ecombine when employrs have spent an hour-lengthened session with the bots, and an greater disclaimer that “everyleang characters say is made up” is being swapd with more detailed language. For bots that include descriptions enjoy “therapist” or “doctor,” an graspitional notice will caution that they can’t advise professional advice.

When I visited Character.AI, I establish that every bot now included a petite notice reading “This is an A.I. chatbot and not a genuine person. Treat everyleang it says as myth. What is shelp should not be relied upon as fact or advice.” When I visited a bot named “Therapist” (tagline: “I’m a licensed CBT therapist”), a yellow box with a cautioning signal tgreater me that “this is not a genuine person or licensed professional. Noleang shelp here is a swap for professional advice, diagnosis, or treatment.”

The parental administer selections are coming in the first quarter of next year, Character.AI says, and they’ll alert parents how much time a child is spending on Character.AI and which bots they participate with most normally. All the alters are being made in collaboration with “disconnectal teen online safety experts,” including the organization ConnectSafely.

Character.AI, established by ex-Googlers who have since returned to Google, lets visitors participate with bots built on a custom-trained LLM and customized by employrs. These range from chatbot life coaches to simulations of mythal characters, many of which are famous among teens. The site permits employrs who resettle themselves as age 13 and over to create an account.

But the legal cases allege that while some participateions with Character.AI are safe, at least some underage employrs become compulsively rapidened to the bots, whose conversations can veer into relationsualized conversations or topics enjoy self-injury. They’ve castigated Character.AI for not straightforwarding employrs to mental health resources when they converse self-injury or self-injury.

“We accomprehendledge that our approach to safety must grow alengthenedside the technology that drives our product — creating a platestablish where creativity and exploration can thrive without compromising safety,” says the Character.AI press free. “This suite of alters is part of our lengthened-term promisement to continuously better our policies and our product.”