After the breakout success of generative AI (GenAI) chatbots three years ago, enterprise deployments are starting to gather momentum. Over three-quarters (78%) of companies now say they use the technology in at least one business function, according to McKinsey. Yet as organizations build out their AI infrastructure, they risk also expanding the AI attack surface.  

AI systems and AI-powered browsers rely on a complex ecosystem of third-party software and components that offer plenty of opportunities for threat actors. To mitigate the reputational, financial and societal damage that security breaches could cause, organizations must redouble their efforts to manage AI supply chain risk. 

Why the Supply Chain is at Risk 

The hyperscalers are estimated to spend nearly $500bn on infrastructure to support AI demand this year. But whenever there’s a race to adopt new technologies, security usually gets treated as an afterthought. It was true of personal computers, the Internet of Things (IoT) and, arguably, cloud computing—resulting in vulnerabilities and misconfigurations eagerly exploited by opportunistic threat actors.  

Unfortunately, these risks also proliferate in the AI supply chain. Threat actors could hijack software update channels to deliver malware, for example. We’ve already observed this happening in East Asia, where adversaries compromised an abandoned update server and used it to host malicious updates. There’s no reason why the same tactics couldn’t be deployed in the AI supply chain.   

AI development also relies heavily on open source components. If an attacker manages to hijack a key library or package, every AI model that uses it becomes vulnerable to compromise. Achieving this could be as simple as socially engineering an open source maintainer—something that’s happening increasingly frequently. 

Yet another risk comes in the form of pre-trained models, which in-house development teams often use to accelerate time-to-value for AI projects. If attackers manage to poison the data in these models and/or hide malicious code such as backdoors in these models, they could be triggered by malicious prompts to alter a model’s output. We’ve detailed multiple examples of real-world incidents like this on platforms such as Hugging Face and GitHub.  

Data poisoning techniques like this can also be used on third-party training datasets, which are readily available on public repositories, and model adapters (like LoRA), which are used to fine-tune LLMs. AI systems might also expose sensitive documents due to incorrectly configured permissions, after accidental or malicious exploitation. AI browser updates and plugins represent yet another risk if they can be compromised. 

Beyond Corporate Risk 

When AI systems are manipulated in these ways, it can be hugely damaging for the companies using them internally or in customer-facing scenarios—potentially leading to data theft, interruption to critical services, or more subtle corruption of decision-making. There are significant financial, reputational and potential compliance implications at play here. But AI supply chain threats are not all about corporate risk. 

Adversarial techniques can be used to bypass the built-in guardrails on many publicly available models, enabling threat actors to use the technology maliciously to create and spread disinformation on a massive scale. So-called “jailbreak-as-a-service” offerings are widespread on the cybercrime underground. The corroding effect of such campaigns is hard to measure, but in many cases, it is designed to undermine the very foundations of democracy. 

In this context, it is concerning that policy around the world trends favor of rapid AI advancement over regulation and safety. AI offers the promise of transforming the way individuals and businesses harness the power of technology. But we can’t be blind to the risks. Prioritizing progress over caution may be seen as a geopolitical necessity, but it could also bring about serious challenges in the future. 

A Pathway to More Secure AI 

There’s no easy way to mitigate risk across such a potentially expansive AI attack surface. But a good place to start would be to secure the data itself and deploy strong controls along zero-trust lines to limit unauthorized access. Some organizations are minimizing their risk exposure by restricting large vendors from using their data for AI model training. 

Secure development and deployment practices can also help, such as inventorying components using a Software Bill of Materials (SBOM), and promptly patching vulnerabilities. Regular security assessments of AI models and applications, via red team exercises, are also a good idea. It goes without saying that pre-trained models should only be obtained from reputable, verifiable sources and subject to integrity checks.  

This is by no means an exhaustive list. If your organization is looking to harness the power of AI, it may want to start first with a single, well-defined project. This enables you to gather any lessons learned and progress from there. Lasting digital transformation success only comes from projects built on a secure foundation.