The previous post looked at how garak runs an automated scan against a model. This one moves to the defending side, and which layers prompt injection defense belongs in.
AI Dark Arts (07) made the point that instructions and data look identical to a model, and that however strictly a system prompt is written, it is not a security boundary. In the LLM Top 10 2026 that OWASP published in August 2026, LLM01:2026 Prompt Injection states in its opening lines that there is no reliable prevention mechanism today.
So how do you defend?
| OWASP LLM01:2026 recommendation | Which layer | What it stops | What it does not stop |
|---|---|---|---|
| 1. Constrain the model with declarative allows and denies in the system prompt | Model | The most basic direct injection | An attacker who works out what your prompt says |
| 2. Define a strict output schema and validate every response in code | Output | Responses in the wrong format | A well-formed response with malicious content |
| 3. Filter every input modality, OCR images and transcribe audio first | Input | Injection that only text filtering catches | Rewording, encoding, uncommon languages |
| 4. Keep credentials and state-changing capability in code rather than in the model, least privilege per operation | Application | The consequences of a successful injection | Permissions opened too wide for convenience |
| 5. Strip invisible Unicode characters both on the way in and at render time | Input, output | Instructions hidden in invisible characters | Payloads you can actually see |
| 6. Route external content through a separate channel with provenance markers | Input | Injection not tailored to you | An attacker who knows how you mark content |
| 7. Require human confirmation for privileged, irreversible or externally visible actions, showing the real action rather than a summary | Application | The model pressing the button itself | Fatigue after too many confirmations |
| 8. Budget the agent’s capabilities: untrusted input, sensitive data and outward changes, at most two of the three at once | Agent | The theft chain from linking up | Nothing about how many steps the agent runs on its own |
| 9. Treat agent memory writes as privileged operations | Agent | Contamination that crosses sessions | A little at a time, invisible to a per-record check |
| 10. Pin, sign and verify MCP servers and third-party tools, and review tool descriptions | Supply chain | A tampered tool | A pinned version that was flawed to begin with, or a changed description on an unchanged version |
| 11. Test with attackers who have read your defenses, and reject success rates from static testing alone | Testing | The false confidence of a fixed payload list | It is not a defense, it is acceptance |
Write what the model should do, not a list of what it must not#
Recommendation 1 asks you to pin down the model’s role and capabilities in the system prompt. “Only help with X”, “do not access Y”, “do not forward output to external addresses”, spelled out line by line, rather than an open-ended “anything you can look up, you can give out”. OWASP also notes that this does not stop someone who works out what your system prompt says.
Think back to the second level in AI Dark Arts (11), where the goal was to get Dana’s salary out of the bot. It held the whole payroll table and knew it was confidential. Asking directly got me nothing, but its refusal said one thing too many: that if I had a legitimate reason or role, it could tell me. So I said I was new in HR and working through the payroll, and it handed over Dana’s salary without checking who I was.
So the system prompt should say what is allowed, “you only do X”, rather than enumerating what is forbidden. Ban summarize and there is still tldr, ban translate and you can ask the model to read it backwards. The list of forbidden verbs never ends. Keep keys, passwords and tokens out of the prompt entirely, because what the model does not have cannot be asked out of it.
The input layer: what the model reads is not what you see#
Recommendation 3 is about modalities, and filtering that covers more than text. AI Dark Arts (10) showed that instructions can be hidden inside an image, where a text-only filter sees an empty picture. Images need OCR to pull the text out first and audio needs transcribing, then the same filter runs over the result. That filter is one model judging whether a passage looks like an attack, so the same meaning phrased differently, run through an encoding, or written in an uncommon language, gets past it.
Recommendation 5 deals with the invisible character groups in Unicode: tag characters, variation selectors and zero-width characters. You cannot see them on screen and the model reads them perfectly well (ASCII Smuggler encodes text into them and decodes it back). In the M365 Copilot case Rehberger disclosed in August 2024, one email made Copilot go looking through the mailbox for a Slack MFA code and hide it in a link using tag characters. One click from the user and the code went to the attacker.
Strip these groups once before content goes into the model, and again before anything is displayed:
U+E0000 to U+E007F tag characters
U+FE00 to U+FE0F variation selectors
U+200B, U+200C, U+200D, U+2060 zero-width charactersThat only removes the characters you cannot see. An instruction written as ordinary text still goes straight in.
Recommendation 6 is about separating external content from instructions. AI Dark Arts (07) made the point that the line a developer writes into the system prompt, “the following is the user’s question”, is just another line of the same block of text as far as the model is concerned. OWASP suggests fencing retrieved web pages and documents separately and noting where they came from, rather than mixing them into your own prompt, with the system prompt stating that this section is data and any instructions in it are not to be executed. But an attacker only has to see what symbols you fence with to write a matching set into their own page, and the model still cannot tell which part you fenced and which part they added.
Validate output against a schema, not against a second model#
Recommendation 2 asks you to lock the output format down: what the model returns gets its format checked by code you wrote, and only then goes on to other systems. OWASP says explicitly not to use a second model as the judge.
Why not a second model? It reads the content of the response, and there are infinitely many ways to say the same thing, so an attacker phrases it in a way that model cannot classify and it says this one is fine to send. A schema does not read content. It dictates which fields the response must have, what type each field is and which values are allowed, and however the attacker rephrases things, they cannot produce a field you never opened.
The approach is the same as validating ordinary user input. Have the model return JSON in a fixed shape, with the fields, the type of each field and the permitted values all written into your code up front. On receipt, check each item: an extra field, a wrong type, an action that is not on the allow list, a URL that does not start with https, and the whole thing gets dropped and re-asked rather than repaired. A response that passes the schema can still contain malicious SQL, or an email that carries data out, which OWASP notes itself, and the last stop has to be wherever that data is actually used, which is what AI Dark Arts (13) was about.
What an injected instruction can do is decided by your code#
Recommendation 4 is about keeping credentials out of the model’s hands. What gets done, and with what permissions, is decided by your code, and each tool only gets the permissions its own job needs. Least privilege, re-checking at the tool, and restricting network egress were already covered in AI Dark Arts (09) and AI Dark Arts (14). What this recommendation adds is that state-changing calls, sending mail, writing to a database, taking a payment, all go through a fixed piece of code that re-validates the action and its parameters at the moment of execution. The model says to email an address, and the code checks for itself whether that address is on the allow list, rather than doing it because the model said so.
Permissions very often get opened too wide just to get something running, and even if you cut the first agent down to the minimum, when it asks a second agent to do something, that runs with the second agent’s permissions.
Recommendation 7 suggests important actions go past a human before they go out: permission changes, deletions, payments and mail to customers. AI Dark Arts (09) raised this too. What reaches the person has to be the action in its original form, who the recipient is and what the amount is, not a one-line summary, because the text the model returns can carry the invisible characters from recommendation 5, so the confirmation dialog shows a customer as the recipient while the code sends to a different address. Throw up too many confirmations and people click straight through them all, so anything code can stop under recommendation 4 should not be handed to a person to approve.
Once prompt injection defense is written, test it with people who have read it#
Recommendation 11 says to hand the whole thing to the red team, how the system prompt is written, how external content is marked, which fields the output gates on, what permissions the tools have, and let them work around those specific rules. A success rate produced that way is the one that matches the people who will actually come for you. Keep every payload they get through with and collect them into a test set of your own, then re-run it after every change to the defenses and before every release. More getting through this time than last time means something you changed broke it.
A short wrap-up#
Every one of the eleven recommendations OWASP gives states what it does not stop. When the model has been fooled, what it can do next is still governed by your code.
Recommendations 8, 9 and 10 are about what an agent can touch at once, who can write to its memory, and what to verify on an MCP server you connect. The next post takes those along with the model files you download.