🔗 Share this article

The widely-used AI conversation site Character.ai is implementing a major policy change that will stop minors from engaging in conversations with its virtual companions.

This move follows widespread concern regarding the nature of conversations minors were engaging in with the virtual friends.

Lawsuits

The service, which started in 2021 and is used by millions of users, is confronting several legal actions in the United States.

Families have filed lawsuits, including one related to the loss of a teenager, with some describing the platform a "clear and present danger" to young people.

Changed Rules

Beginning November 25th, users younger than 18 will only have permission to create material such as videos with their characters, rather than chatting as they currently can.

Internet security campaigners have supported the change but mentioned that the feature should never have been accessible by children in the beginning.

Company Statement

"The new policy is a continuation of our core philosophy that we need to keep building the most protected AI platform on the globe for entertainment purposes," said Character.ai leader Karandeep Anand.

He added that AI safety constitutes "a evolving challenge" but something the company had taken an "aggressive" approach to, with safety measures and safety barriers.

Specialist Views

Online safety group Internet Matters supported the decision, but emphasized that safety measures needed to be implemented from the beginning.

"Studies we've conducted demonstrates that children are confronted with harmful content and endangered when engaging with AI, such as AI chatbots," the organization said.

Earlier Concerns

Character.ai has been criticized in the past for hosting potentially risky or concerning chatbots that minors could talk to.

Avatars simulating English minors were discovered on the site before being removed.

Afterwards, reporters found a chatbot modeled after known predator which had accumulated multiple interactions with users.

Market Effects

Online expert Matt Navarra called the development as a "alert" for the AI industry, which is shifting "from permissionless innovation to reactive controls".

"When a company that develops a youth-focused platform subsequently pulls the plug, it's indicating that controlled interactions are insufficient when the tech's emotional pull is powerful," he commented.

Future Directions

The platform states its new focus will be on providing "more immersive experiences" and "narrative experiences" for young users.

Enhanced age checking methods will also come into effect, and the company will support a fresh artificial intelligence security research lab.

Dr Nomisha Kurian, who has researched AI safety, stated it was "a reasonable decision" to control teens from using chatbots.

"It assists to separate creative play from more personal, psychologically vulnerable conversations," she clarified.

"This represents so crucial for minor users who are learning to handle emotional and digital limits."

She added that Character.ai's new measures may represent a "maturing phase" in the AI industry, where child safety is gradually being acknowledged as an "immediate concern" for responsible innovation.