A CISO pointed me at it. The attack hides instructions inside an image. White text on a white background, or characters skewed enough that a scanner sees garbage while the model reads them fine. The attacker leaves the image in a public repository. Someone asks their agent to deploy the site. The agent reads the image the way it processes everything else, follows the hidden instruction, and quietly creates an admin account the attacker controls. The site deploys correctly. The text guardrails stay green.
What broke was the entry. My trust tagging was built for words. A retrieved image carried no label at all.
The executor is supposed to refuse any step where untrusted content drives a privileged action. That rule reads two things, where the content came from and what the step would do. It never reads the instruction, which is why it does not care whether the attack arrived as text, an image, or memory.
But it needs a label, and the image had none. So the rule had nothing to fire on, and what stopped the step was that my tool registry has no "create admin account" tool. That is luck, not architecture.
Screening is an arms race and enforcement is not. Finding every hidden instruction in every image forever is a problem I lose. Untrusted input cannot drive a privileged action, whatever channel it came through, is a rule I can write down and test. Build the barrier first and the screening does not have to be perfect.
Tag every input channel at ingestion, not just the text one. Classify a tool that fetches a URL as something that sends data, because that is what it does. Both extend the design instead of replacing it.
When did you last run a new attack class against your own architecture, and what did it find?
Written by Duane Grey
AI Strategy & Implementation
Independent AI consultant helping companies cut through hype and deploy systems that produce real results.