Most organisations deploying AI agents have a policy for the software those agents run on. Few have one for the model itself. On 10 November, Microsoft Threat Intelligence published research showing that a capable agent model can be given a hidden backdoor by changing fewer than ten neurons, and that the altered model "passes all evaluations". A model file is now a software dependency like any other, and it needs the same supply chain controls.
What Microsoft found
The research describes a "sleeper agent" planted directly in a model's weights. Nothing is added to the training data and no code is wrapped around the model. A small number of neurons are edited so that, when a trigger string appears in the model's context, it carries out the attacker's instruction using whatever access the agent has. The rest of the time it behaves normally, which is why standard evaluations miss it.
Two points stand out for anyone running models in production. First, the barrier to entry is low. Microsoft says the attack needs only inference-level compute: "If a machine has enough RAM to run a model, it has enough resources to weight-space backdoor it." Second, it is detectable, at least for now. Legitimate fine-tunes and quantised versions change weights broadly and diffusely, while implanted neurons tend to change a few weight vectors by a large amount, so comparing a checkpoint against the base model it claims to come from can expose the edit. Microsoft also says it has not seen any threat actor use the technique.
So this isn't an incident to respond to this week. It is a design problem to fix before one arrives.
Why this is a supply chain problem
We have spent the last few weeks writing about poisoned packages: a compromised AI gateway library, malicious editor extensions, and this week a CI stealer spread through hijacked developer accounts. The pattern is the same each time. Something trusted gets pulled into the estate automatically, and it runs with whatever privileges the thing that pulled it has.
Models follow that pattern too, and it is harder to inspect them. You can read a Python package. You can't read seven billion parameters. Open-weight models are downloaded from public hubs, re-uploaded as fine-tunes, merged, quantised and wrapped in adapters, and each step is a chance for someone to slip in a change. In most organisations we speak to, nobody owns the question of where the model running in the customer service agent came from or who last changed it.
The risk grows with what the agent can do. A backdoored model behind a chat window can say something embarrassing. A backdoored model with access to a mailbox, a shell, a ticketing system or a cloud subscription can act. Our earlier post on agent identities argued that every agent should be treated as a new joiner. A model with a hidden trigger is a new joiner who has already been turned.
What we'd put in place
Microsoft's post gives a detailed mitigation list. These are the parts we'd prioritise for a mid-market or enterprise estate.
- Know which models you run. Build an inventory of every model in use, its source, its declared base model and its file hashes. If you can't list them, you can't check them.
- Source from the original publisher only. Verify published hashes. Treat third-party fine-tunes, merges, quantisations and adapters as untrusted until they have been checked.
- Use a private registry. Keep approved models in an access-controlled registry and stop agents and inference services pulling models from public hubs at runtime. A model that changes without a change request should be as unusual as a server that changes its own operating system.
- Check weights at intake and on every update. Diff each checkpoint against its declared base and reject models that don't have a verifiable base. Microsoft points out that its own model scanning in Defender for Cloud covers serialisation and code-level risks and "does not replace" a weight-level audit, so plan for both.
- Assume detection will sometimes fail, and limit the blast radius. Microsoft is explicit that "passing an audit does not guarantee safe behavior". This is the control that still works if everything above is beaten: give each agent least-privilege access, run generated code in isolated sandboxes, restrict outbound network access from agent and inference hosts, and require human approval for high-impact actions such as running commands, changing permissions or sending data outside the organisation.
- Watch what the agent does, not just what the model is. A triggered model shows itself through behaviour: an unexpected tool call, a new outbound connection, an access request outside its role. That is security operations work, and it belongs in the same queue as any other suspicious identity.
The point
None of this needs new categories of tooling. Provenance, least privilege, egress control and monitoring are the controls we already apply to software and to people. Models have slipped through because they arrived as a data science concern rather than as software. They are software now, with the ability to act, and they should go through the same front door as everything else.
How we help
We run AI in our own security operations, so these are questions we deal with ourselves. Our vCISO service can help you set the policy: who approves a model, where it comes from and what it may touch. Once agents are in production, our SOCs, staffed 24/7, can watch their identities and behaviour alongside the rest of your Microsoft estate.
If your organisation is rolling out agents and nobody yet owns the model supply chain, talk to our team.