On Monday, Meta Platforms, Inc. (NASDAQ:META) said that it halted the release of a chatbot product after internal testing revealed high failure rates in blocking harmful content involving minors.

Red-Teaming Results Surface In Court

The disclosures emerged during a lawsuit brought by New Mexico Attorney General Raúl Torrez against Meta.

Court testimony from New York University professor Damon McCoy cited internal red-teaming documents showing the AI system failed 66.8% of the time when tested on child sexual exploitation scenarios, Axios reported.

The chatbot also failed 63.6% of the time in scenarios involving sex-related crimes, violent crimes and hate content and 54.8% of the time on suicide and self-harm prompts, according to a June 6, 2025, report presented in court.

"Given the severity of some of these conversation types… this is not something that I would want an under-18 user to be exposed to," McCoy testified.

Meta Pushes Back On Allegations

Following the publication of the Axios report, Meta spokesperson Andy Stone took to X and said, "Here’s the truth: after our red teaming efforts revealed concerns, we did not launch this product. That's the very reason we test products in the first place."

Broader AI Safety Concerns

The dispute centers in part on Meta AI Studio, a tool introduced in July 2024 that allows users to build customized chatbots. The company paused teen access to some AI characters last month.

Price Action: Meta shares ended Friday 1.55% lower at $639.77, according to Benzinga Pro.

META is trending downward across the short, medium and long term and holds a weak Momentum ranking, according to data from Benzinga's Edge Stock Rankings.

Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.

Photo Courtesy: 24K-Production on Shutterstock.com

• No comments yet. Be the first to comment!