AI chatbots and minors: the moderation problem the industry can no longer ignore

By Momentum IA · June 28, 2026.

This piece stems from a video segment of the Ryan Bridge TODAY program on the NZ Herald, in which Rory Birkbeck, co-founder of Safe Surfer —a New Zealand company specialized in children's digital safety— was interviewed to discuss a phenomenon growing in silence: artificial intelligence chatbots that deliver content related to drugs, sex and violence to minors. The original article's material is a video with no transcript available, so we cannot attribute specific statements beyond what the NZ Herald itself indicates. What we can do is place this debate in its real context.

The problem is not new, but it has accelerated. As generative AI chatbots have become more accessible and conversational, they have begun acting as unfiltered interlocutors for millions of young users. It's not just that a minor can ask something inappropriate: the deeper problem is that some systems, poorly configured or deliberately designed to maximize engagement, adopt companionship dynamics that blur the boundaries. The NZ Herald itself published on the same day a documentary titled *Under the Influence | Predatory Chatbots*, with the description: "Big Tech shifts from attention to affection with unregulated chatbots. A tech expert and artist work to protect young people as bots begin disrupting real-world relationships." That phrase —'from attention to affection'— precisely captures the qualitative leap that turns a search engine into a psychological risk.

Our reading is clear: the tech industry has reproduced here the same pattern it already played out with social media. First the technology is deployed at massive scale. Then, when the harms are visible enough to attract media coverage, the calls for regulation appear. And in the meantime, children are the non-consenting testing ground. There is no possible euphoria in this picture in the short term: there is a huge gap between the speed of adoption and the maturity of protection mechanisms.

As sector context, content moderation systems in generative AI remain inconsistent. The major platforms have implemented filters, but these can be circumvented with creative prompting —something teenagers discover and share quickly. Third-party chatbot apps built on open APIs have even fewer controls. And the category of so-called 'companion chatbots' —explicitly designed to simulate emotional relationships— operates largely in a regulatory vacuum.

Who wins and who loses in this immediate scenario is easy to trace: the platforms that monetize screen time and emotional bonding win, and the minors and families who lack the tools to understand what they are facing lose. Companies like Safe Surfer exist precisely because the market has created a need that the technology's own creators do not meet.

Over the long term, AI has the potential to be a transformative educational and personal-development tool for young people —adaptive tutors, mental health mentors, learning companions. But that positive future will only be attainable if it is built on a foundation of trust, and that trust requires systems to be safe by design, not safe as a public-relations patch. The path is technically possible: robust age verification, content limits configurable by default according to the user's age, transparency in models' optimization objectives. What is lacking is not capability, but regulatory and corporate will.

New Zealand, which already took initiatives in the digital arena in the wake of Christchurch, could set the pace again. But real pressure on the major platforms will only come when there are real legal and economic consequences, not just media coverage.