NCA warns parents over AI-generated child abuse material risk

The National Crime Agency has issued a public warning urging parents not to share children's images online, citing a growing threat of those images being manipulated using AI to create child sexual abuse material. The BBC reports that the NCA's warning comes amid increasing evidence that publicly available photographs of children are being harvested and altered using generative AI tools. The agency is advising families to review privacy settings on social media platforms, limit who can view images of children, and consider whether sharing certain photographs is necessary. The warning reflects broader concerns about how accessible AI tools are lowering the technical barrier for creating harmful content, and how widely shared family images can be repurposed without consent or knowledge.

Why this matters for UK organisations

This development is relevant beyond safeguarding policy. It highlights how generative AI is being used to manipulate legitimate content in ways that create reputational, legal and safeguarding risks. Organisations that handle images of children, whether in education, healthcare, sports clubs, charities or local authorities, should review how those images are stored, shared and published. The operational question is whether image sharing practices, consent processes and privacy controls reflect the changed risk environment. This is also a prompt to consider how AI acceptable use policies address the creation or manipulation of images, and whether staff understand the safeguarding implications of generative AI tools that may be accessible through personal or work devices. The NCA's warning underscores that the risk is not theoretical: publicly available images are being actively harvested and misused, and organisations have a responsibility to consider how their own practices may inadvertently contribute to that risk.

What to review

Organisations should review image handling policies and consent frameworks to ensure they reflect the current risk environment. This includes checking privacy settings on websites and social media accounts, reviewing who has access to image libraries, and ensuring that staff understand the safeguarding risks associated with generative AI. It is worth considering whether acceptable use policies adequately cover the creation or manipulation of images using AI tools, and whether training on safeguarding and data protection includes guidance on these emerging risks. For organisations that publish images of children, this is a prompt to review whether those images are necessary, whether consent is appropriately documented, and whether privacy controls are proportionate to the risks involved.

Source: BBC Technology