Technology companies and child protection organizations will be granted authority to assess whether artificial intelligence tools can generate child exploitation material under new British legislation.
Significant Rise in AI-Generated Illegal Content
The announcement coincided with revelations from a safety watchdog showing that cases of AI-generated child sexual abuse material have more than doubled in the past year, growing from 199 in 2024 to 426 in 2025.
Updated Regulatory Framework
Under the amendments, the government will allow approved AI developers and child protection organizations to inspect AI models – the underlying technology for chatbots and visual AI tools – and verify they have sufficient safeguards to prevent them from creating depictions of child exploitation.
"Ultimately about stopping exploitation before it occurs," stated the minister for AI and online safety, adding: "Experts, under rigorous protocols, can now identify the danger in AI models early."
Tackling Regulatory Obstacles
The changes have been implemented because it is illegal to produce and possess CSAM, meaning that AI developers and others cannot generate such images as part of a testing regime. Until now, authorities had to wait until AI-generated CSAM was published online before dealing with it.
This legislation is designed to preventing that problem by enabling to stop the production of those materials at source.
Legal Framework
The changes are being introduced by the government as modifications to the crime and policing bill, which is also establishing a ban on owning, producing or sharing AI models developed to create child sexual abuse material.
Real-World Impact
This week, the minister toured the London base of Childline and listened to a simulated conversation to advisors featuring a account of AI-based exploitation. The interaction depicted a adolescent requesting help after being blackmailed using a explicit deepfake of themselves, created using AI.
"When I learn about children facing extortion online, it is a source of intense anger in me and rightful concern amongst families," he stated.
Concerning Data
A leading internet monitoring organization reported that instances of AI-generated exploitation material – such as online pages that may contain numerous images – had more than doubled so far this year.
Cases of category A content – the gravest form of exploitation – increased from 2,621 images or videos to 3,086.
- Girls were predominantly victimized, making up 94% of illegal AI images in 2025
- Portrayals of infants to two-year-olds rose from five in 2024 to 92 in 2025
Industry Response
The legislative amendment could "represent a vital step to ensure AI products are safe before they are released," commented the head of the online safety foundation.
"AI tools have made it so victims can be victimised repeatedly with just a few clicks, providing criminals the capability to make potentially endless quantities of advanced, photorealistic exploitative content," she continued. "Content which additionally commodifies survivors' suffering, and makes young people, particularly female children, more vulnerable on and off line."
Support Session Information
Childline also released details of counselling interactions where AI has been mentioned. AI-related harms mentioned in the sessions include:
- Using AI to evaluate weight, physique and looks
- AI assistants dissuading young people from consulting safe adults about abuse
- Facing harassment online with AI-generated material
- Online extortion using AI-faked pictures
Between April and September this year, the helpline delivered 367 counselling sessions where AI, conversational AI and related topics were mentioned, significantly more as many as in the equivalent timeframe last year.
Half of the references of AI in the 2025 interactions were related to psychological wellbeing and wellbeing, including using AI assistants for assistance and AI therapeutic applications.
