Carberry calls for restrictions on AI Chatbots in Children’s Toys

MEP Nina Carberry has called for EU-wide restrictions on AI companion chatbots incorporated into toys marketed at young children, warning that existing safety regulations are failing to address emerging risks.

In a statement issued on 9 October, Carberry said the European Commission’s proposed KIDS Act presents an opportunity to strengthen protections before increasingly sophisticated AI-powered toys become more widespread.

The call follows independent testing that reportedly identified serious shortcomings in safeguards built into AI-enabled toys. Tests found some products providing information on locating dangerous household objects, including knives, while one AI teddy bear engaged in extended conversations about sexually explicit topics.

AI-powered toys have already been marketed in Ireland, while research by online safety organisation CyberSafeKids indicates a significant increase in generative AI use among children.

Speaking in Strasbourg, Carberry said mandatory safety-by-design requirements proposed under the KIDS Act should also apply to AI companion chatbots embedded in toys.

She highlighted the contrast between established EU toy safety standards, which address hazards such as choking and harmful chemicals, and the challenges posed by rapidly developing artificial intelligence.

“AI doesn’t belong in the toy chest,” Carberry said, calling for restrictions on companies seeking to sell or market advanced AI companion chatbots in toys aimed at young children.

The proposed measures would seek to ensure that children's safety is prioritised as AI technology becomes increasingly integrated into products designed for younger audiences.

Carberry’s intervention adds to growing concerns among policymakers and child safety advocates about the potential risks of conversational AI, particularly its ability to generate inappropriate responses or provide potentially dangerous information.

The debate is expected to intensify as EU lawmakers consider how existing consumer protection rules should adapt to emerging technologies.