🔗 Share this article

Technology companies and child protection agencies will receive permission to assess whether AI systems can produce child abuse material under recently introduced UK laws.

Substantial Increase in AI-Generated Illegal Material

The announcement coincided with revelations from a protection monitoring body showing that cases of AI-generated child sexual abuse material have increased dramatically in the past year, growing from 199 in 2024 to 426 in 2025.

Updated Regulatory Structure

Under the amendments, the authorities will permit designated AI companies and child safety groups to examine AI models – the foundational technology for chatbots and visual AI tools – and verify they have sufficient protective measures to stop them from producing depictions of child sexual abuse.

"Fundamentally about preventing exploitation before it happens," declared Kanishka Narayan, noting: "Specialists, under rigorous protocols, can now identify the danger in AI systems promptly."

Tackling Regulatory Challenges

The changes have been implemented because it is against the law to produce and possess CSAM, meaning that AI creators and others cannot generate such images as part of a evaluation process. Until now, authorities had to delay action until AI-generated CSAM was published online before dealing with it.

This law is aimed at preventing that issue by enabling to halt the creation of those images at their origin.

Legislative Structure

The changes are being added by the government as revisions to the crime and policing bill, which is also establishing a ban on owning, producing or sharing AI models developed to generate exploitative content.

Real-World Consequences

This week, the official toured the London base of Childline and listened to a mock-up call to counsellors featuring a account of AI-based exploitation. The call portrayed a teenager requesting help after facing extortion using a explicit deepfake of himself, created using AI.

"When I learn about young people facing blackmail online, it is a source of intense anger in me and rightful concern amongst families," he said.

Alarming Statistics

A leading online safety organization reported that instances of AI-generated exploitation content – such as webpages that may include multiple images – had more than doubled so far this year.

Cases of category A content – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Female children were predominantly victimized, making up 94% of illegal AI depictions in 2025
• Depictions of infants to two-year-olds rose from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "represent a crucial step to guarantee AI products are secure before they are released," commented the chief executive of the online safety foundation.

"AI tools have enabled so victims can be targeted all over again with just a few clicks, providing offenders the ability to make potentially endless quantities of advanced, photorealistic child sexual abuse material," she continued. "Material which additionally exploits victims' suffering, and renders children, especially female children, more vulnerable on and off line."

Counseling Interaction Data

Childline also released details of support sessions where AI has been referenced. AI-related risks mentioned in the conversations comprise:

• Using AI to rate body size, body and appearance
• Chatbots dissuading children from talking to trusted guardians about abuse
• Facing harassment online with AI-generated material
• Online extortion using AI-faked images

During April and September this year, Childline delivered 367 support interactions where AI, chatbots and associated terms were mentioned, four times as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 sessions were related to psychological wellbeing and wellness, including using chatbots for support and AI therapeutic apps.