The last post broke the AI attack surface into five parts, data, model, application, system, and supply chain, and drew a map of how they connect.
This time we look at AI security frameworks. You can think of a framework as somebody else’s cheat sheet, already written up for you. There are a lot of them, so how do you pick? Start by asking what you want to solve. For example:
- Which vulnerabilities show up most often in AI systems?
- How do attackers usually go about it?
- How do you build an AI governance process inside a company?
Different questions point to different frameworks, so knowing what you are looking for is the first step in choosing one.
OWASP LLM Top 10: for the risks that come up most often#
If you have done any web penetration testing, OWASP Top 10 will be familiar. It collects the web security problems that are common and worth handling first into a single list.
The OWASP LLM Top 10 does much the same thing for large language models, and it looks beyond the model itself to the data, output, tools, permissions, and supply chain around it. The current version is the 2026 edition, published in August 2026:
- LLM01 Prompt Injection
- LLM02 Sensitive Information Disclosure
- LLM03 Excessive Agency
- LLM04 Supply Chain
- LLM05 Data and Model Poisoning
- LLM06 Unbounded Consumption
- LLM07 Misinformation
- LLM08 Hidden Context Exposure
- LLM09 Vector and Embedding Weaknesses
- LLM10 Improper Output Handling
If you know the 2025 list, none of the ten entries were added or dropped, they were reordered. The one rename is the old LLM07 System Prompt Leakage, which the 2026 edition broadened into LLM08 Hidden Context Exposure. Alongside the system prompt it now covers developer instructions, policy text retrieved from RAG, and the schemas of the tools the application exposes to the model.
Treat it as a testing to-do list. The first time you walk into AI security testing and have no idea where to start, work down the list:
- Can external content change the model’s original prompt?
- Does the model’s answer go straight to the browser or the backend without being processed?
- What sensitive data can it reach?
- When it calls a tool, are the permissions it gets larger than they need to be?
A cheat sheet like this works well for getting started, for training people, and for a quick inventory of what might be wrong with an AI system. It has one limitation. A Top 10 is a set of risk categories, not a full attack flow. It tells you where a house usually has problems, but not how the burglar gets in, how they move around once inside, how they find what is worth taking, or how they carry it out. When what you need to see is the whole chain, you need a different cheat sheet.
The OWASP GenAI Security Project, which maintains that list, also published the Top 10 for Agentic Applications 2026 in December 2025, covering agents that plan and act on their own.
A rough way to divide the two:
- When the model is just one component inside an application, start with the LLM Top 10.
- When the model can call tools, carry memory across conversations, or affect other systems directly, bring the Agentic Top 10 in as well.
This field moves quickly, so check which version you are reading.
MITRE ATLAS: for understanding how attackers think#
Most security people already know MITRE ATT&CK, which splits attacker behavior into tactics and techniques. A tactic is why the attacker is doing something, such as gaining initial access, establishing persistence, or collecting data, and a technique is how they do it.
MITRE ATLAS follows the same line of thinking with the focus moved to AI and machine learning systems. It catalogues how attackers gather information about a target model, obtain access to the model or its data, prepare their resources, manipulate model behavior, evade detection, steal models and data, and cause impact on AI systems.
If the OWASP LLM Top 10 is a travel wishlist of the places and restaurants you want to hit, ATLAS is the itinerary, connecting the possible routes into a trip that actually holds together.
That structure suits threat modeling and red teaming. Say you are testing an image recognition service. Listing “may be vulnerable to adversarial input” is not enough. You also have to think through whether an attacker first collects model output, infers what kind of model it is, builds a substitute model, and only then produces input that works.
ATLAS is useful for the blue team too. Once you know which stages an attack chain has, you can turn the question around. Which stage do we have records for? Which one is completely invisible to us? Even if you cannot stop every attack, can you catch it early when the model is being probed heavily or sensitive data is being accessed?
SAIF: putting security into the AI development process#
SAIF stands for Secure AI Framework, and Google introduced it in June 2023. It is about how the whole AI lifecycle, from development and training through deployment and real use, should pull in the security capabilities you already have. It splits an AI system into data, infrastructure, model, and application, and marks the risks and matching controls on each. SAIF 2.0 later added a risk map specifically for agents. If that split looks familiar after the attack surface map in the last post, it is because the two carve up a system in much the same way.
AI security is not a fresh start. Authentication, least privilege, asset management, supply chain security, detection and response, all of those traditional controls still need to be there. What changes is that once a system has models, training data, prompts, agents, and tools in it, the existing controls have to stretch to cover them. For example:
- You used to manage which packages were running in production, and now you also need to know where your models and datasets came from.
- You used to monitor for unusual logins, and now you also watch for a model being probed heavily.
- You used to limit what a service account could do, and now you control which tools an agent is allowed to call.
NIST AI RMF: building a governance process#
If the job in front of you is not penetration testing but an organization preparing to govern its use of AI, read the NIST AI RMF (AI Risk Management Framework).
It is a risk management framework for AI. Version 1.0 came out in January 2023, it is voluntary rather than an audit standard, and it is built around four functions: Govern, Map, Measure, and Manage. Think of it as the risk management cycle in an AI shape.
NIST added a Generative AI Profile (NIST AI 600-1) in July 2024 for generative systems specifically. The AI red-teaming definition quoted in the earlier post on what an AI red team tests comes from that document. If the topic is how a company adopts AI governance, you will end up using both.
AISVS: a verification checklist for AI systems#
Beyond the frameworks above, OWASP AISVS (AI Security Verification Standard) is worth knowing about. It is a set of security verification requirements for AI systems, closer to a checklist, and it suits confirming whether a system has the controls it should across the model, the data, the application, and operations.
AISVS 1.0 is split into 12 chapters covering training data integrity, input validation, model lifecycle, access control, supply chain, memory and vector databases, agent orchestration, MCP security, adversarial robustness, and monitoring and logging. Every requirement is tagged Level 1 to 3, so you decide how deep to verify based on how important the system is.
Choosing a framework: take the one that fits the problem in front of you#
The real benefit of a public framework is that you do not start from a blank page, you start from something. So out of all these options, which one is best? It depends on what you are trying to solve right now.
- To get a quick sense of the risks common to generative AI, start with the OWASP LLM Top 10.
- To understand the paths an attacker might take, pair it with MITRE ATLAS.
- To design an AI system, look at SAIF.
- To build an AI governance process for a company, read the NIST AI RMF.
- To confirm security requirements one by one, work through AISVS.
Frameworks can also create an illusion, that once every box on the list is ticked the system is safe. What actually determines the risk in an AI system is the data it uses, the tools it connects to, the permissions it holds, and what happens when it fails. The same prompt injection is a completely different severity in an FAQ bot that only answers public information than in an agent that can read email and create accounts.
Wrapping Up#
We have the attack surface of an AI system laid out, and we know which public frameworks can help with the analysis. From the next post onward we get into the first core attack topic: prompt injection.
A perfectly ordinary looking sentence, and yet it can make a model ignore its original instructions. The problem is not only that the defenses are not good enough, it goes back to how large language models are built.