A concerning trend has emerged in the UK, where children are increasingly reporting explicit deepfake images featuring their likenesses. The Report Remove service, designed to tackle such issues, has already logged 420 reports from young individuals in just the first half of this year—surpassing the total from all of 2025. This alarming rise has intensified discussions about the implications of artificial intelligence (AI) in creating harmful digital content.
An Alarming Increase in Deepfake Reports
The Report Remove service, which operates in conjunction with the Internet Watch Foundation (IWF) and the NSPCC, is a crucial tool for children wishing to report intimate images that have been manipulated without their consent. This year alone, the service has recorded a staggering increase in reports of altered images, with 420 cases already filed. This figure not only highlights the severity of the issue but also indicates a distressing rise from the previous year’s total of 397 reports.
Dan Sexton, the IWF’s chief technology officer, emphasises the challenge posed by the accessibility of sophisticated AI tools. “If someone can take a benign image of a child from the internet and create child sexual abuse material with it, there is nothing that child can do to stop it from happening,” he warns. This statement underscores the vulnerability of children in an increasingly digital world.
The Role of AI in Creating Harmful Content
The capabilities of AI have raised significant concerns, especially when it comes to the manipulation of images to produce explicit content. Sexton points out that the prevalence of AI-generated material has become a grim reality. “This is one of the harms we were worried about when we first saw AI image generators being used,” he explains, stressing the urgent need for protective measures.
The IWF has noted that many of the reports submitted this year qualify as child sexual abuse material (CSAM), which is illegal in the UK. The government is taking steps to combat this issue by outlawing the adaptation of AI models for creating such content. However, experts like Rani Govender from the NSPCC argue that tech companies must do more to integrate robust safety features into their products to mitigate the risk of explicit material being generated at all.
Sextortion: A Disturbing Consequence
The rise in deepfake incidents has not only led to the creation of harmful content but has also facilitated dangerous scenarios like sextortion. Some victims who sought help from Report Remove have reported being targeted by blackmailers, who threaten to publish manipulated images unless they receive financial compensation or further explicit material. This disturbing trend highlights the broader implications of deepfake technology and the urgent need for protective measures.
Report Remove operates by allowing children to submit reports anonymously, turning flagged images into a digital fingerprint or “hash”. This fingerprint is then shared with major tech platforms, enabling them to remove or prevent the upload of such content. However, as the number of reports continues to rise, it raises questions about the effectiveness of current measures and the urgency for enhanced protections.
Guidance for Parents and Guardians
In light of these developments, the National Crime Agency and IWF have issued urgent guidelines for parents and guardians. They recommend keeping identifiable images of children private, suggesting that social media accounts be secured or shared only with close friends. These precautions aim to protect children from the growing threat of cybercriminals who scrape online images to create deepfakes for malicious purposes.
Schools have also been advised to remove identifiable images of students from their websites and social media platforms to reduce the risks associated with AI-generated CSAM. This proactive approach is essential in safeguarding the interests of children in an era where digital content can be manipulated with alarming ease.
Why it Matters
The surge in reports of deepfakes involving children reveals a pressing need for enhanced digital safety measures. As the capabilities of AI technology continue to evolve, so too must the strategies employed by parents, educators, and tech companies to protect vulnerable users. This crisis highlights the critical nature of online safety in an age where the line between reality and digital fabrication is increasingly blurred. The time for action is now, as the well-being of future generations hangs in the balance.
