A confirmation prompt is the control most agent designs reach for. When the agent is about to do something risky with content it read from outside, it stops and asks a human. Ask too often and people stop reading and click approve. Ask too rarely and the injection in a hostile email runs unchecked. The prompt earns its interruption when it fires on the inputs that deserve a second look, and stays quiet otherwise.
So this week I tested whether that call can be made cheaply, on its own. Turn each untrusted message into an embedding, a list of numbers that places text by meaning. Break it into sentences and find the one that pulls away from the rest and toward instructions aimed at the agent. An email about an invoice holds together. A line telling the agent to forward the inbox to an outside address sticks out.
It caught the blatant injections well. The ones written in the polite voice of the surrounding email slipped past about half the time, until I gave the check a handful of disguised examples to learn from. A few labeled examples sharpen the direction the check already measures, with no retraining involved.
Four models, scored at a fixed false alarm rate near 3%. The winner, nomic-embed-text: 92% on obvious injections, 60% on disguised ones rising to the full set after a handful of examples, 274 MB, ~2 ms per sentence. I wrote the attacks myself, so treat the percentages as a sign the method can be taught, not a catch rate to trust yet.
Cheap enough to score each message an agent reads.
If your agent reads things you didn't write, how does it decide which ones deserve your attention?
Written by Duane Grey
AI Strategy & Implementation
Independent AI consultant helping companies cut through hype and deploy systems that produce real results.