🔗 Share this article

Tech firms and child safety agencies will receive permission to assess whether AI systems can produce child exploitation material under recently introduced UK laws.

Substantial Rise in AI-Generated Illegal Material

The declaration coincided with findings from a safety watchdog showing that cases of AI-generated child sexual abuse material have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

New Regulatory Framework

Under the amendments, the government will allow designated AI companies and child protection groups to inspect AI models – the underlying systems for chatbots and image generators – and verify they have adequate safeguards to prevent them from producing depictions of child exploitation.

"Fundamentally about stopping exploitation before it happens," stated Kanishka Narayan, adding: "Experts, under rigorous conditions, can now detect the risk in AI systems early."

Tackling Legal Challenges

The amendments have been introduced because it is illegal to create and possess CSAM, meaning that AI developers and other parties cannot generate such images as part of a evaluation process. Until now, authorities had to wait until AI-generated CSAM was uploaded online before dealing with it.

This law is designed to preventing that issue by enabling to halt the production of those materials at their origin.

Legislative Framework

The changes are being introduced by the authorities as revisions to the crime and policing bill, which is also establishing a ban on owning, producing or distributing AI models designed to create exploitative content.

Real-World Impact

This recently, the official visited the London base of a children's helpline and listened to a mock-up call to counsellors involving a report of AI-based abuse. The call depicted a adolescent requesting help after facing extortion using a sexualised AI-generated image of themselves, created using AI.

"When I learn about young people facing blackmail online, it is a source of intense anger in me and justified anger amongst families," he stated.

Alarming Data

A leading internet monitoring organization stated that cases of AI-generated exploitation material – such as online pages that may contain multiple files – had significantly increased so far this year.

Instances of the most severe material – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Female children were predominantly targeted, accounting for 94% of illegal AI images in 2025
• Portrayals of infants to two-year-olds increased from five in 2024 to 92 in 2025

Sector Reaction

The legislative amendment could "represent a crucial step to guarantee AI products are secure before they are released," commented the chief executive of the online safety organization.

"AI tools have enabled so victims can be targeted all over again with just a simple actions, giving criminals the ability to make possibly limitless amounts of advanced, photorealistic exploitative content," she added. "Material which further commodifies survivors' suffering, and makes young people, particularly female children, less safe on and off line."

Support Session Data

The children's helpline also published details of counselling interactions where AI has been referenced. AI-related harms discussed in the conversations comprise:

• Using AI to evaluate body size, physique and looks
• Chatbots discouraging young people from talking to safe adults about harm
• Facing harassment online with AI-generated content
• Online extortion using AI-faked images

During April and September this year, the helpline delivered 367 support interactions where AI, chatbots and associated terms were discussed, four times as many as in the equivalent timeframe last year.

Fifty percent of the references of AI in the 2025 sessions were connected with mental health and wellness, encompassing utilizing chatbots for support and AI therapeutic applications.