AI Coding Tools Consistently Hallucinate Same Fake Package Names, Creating Supply Chain Risk

Research by Aleksandr Churilov has found that the top AI coding assistants are remarkably consistent in hallucinating the same non-existent software package names, creating predictable supply chain vulnerabilities through a technique known as slopsquatting. Churilov identified 127 fake package names that were repeatedly suggested by five different large language models when developers asked for help with coding tasks. Attackers are exploiting this by creating malicious packages with these hallucinated names on public repositories such as PyPI and npm, knowing that developers using AI coding tools are likely to be directed towards them. This represents a new form of supply chain attack where the predictability of AI hallucinations allows attackers to pre-position malicious code in locations where developers are statistically likely to look.

Why this matters for UK organisations

For UK organisations using AI-assisted development tools, this creates a tangible and growing risk to software supply chain integrity. Developers increasingly rely on AI coding assistants to speed up development, suggest libraries and generate boilerplate code, but these tools do not verify whether the packages they recommend actually exist or are safe to use. The consistency of hallucinations across multiple AI models means attackers can systematically create malicious packages that will be recommended to developers at scale. This is particularly concerning for organisations where developers have autonomy to add dependencies without rigorous review, or where development environments are not isolated from production systems. The risk is compounded by the fact that many developers trust AI-generated suggestions without verification, especially when under time pressure or working on unfamiliar codebases. For organisations in sectors such as finance, healthcare and government, where software supply chain integrity is critical, this represents a significant and evolving risk that requires clear policy, process and technical controls.

What to review

Organisations should review whether their software development lifecycle includes verification steps to ensure that dependencies suggested by AI tools are legitimate, actively maintained and from trusted sources before they are introduced into development or production environments. This includes implementing dependency scanning tools that flag unknown or suspicious packages, and ensuring that code review processes include checks for dependencies that may have been added based on AI suggestions. Developers should be made aware of the risks of blindly trusting AI-generated code suggestions, and should be trained to verify package names, check repository activity and review package maintainers before adding new dependencies. Build pipeline security controls should be reviewed to ensure they can detect and block malicious or hallucinated packages before they reach production. Organisations should also consider whether development environments are sufficiently isolated from production systems, and whether dependency management policies are clear about who is responsible for approving new packages. Finally, organisations should clarify who owns software supply chain security, and ensure that policies, training and technical controls are regularly reviewed and updated as AI coding tools become more widely adopted.

Source: CSO Online