In this series, I will break down the attack surface of AI systems layer by layer. I will start with generative AI, prompt injection, agents, and MCP, and then move on to attacks at the data, model, and supply chain layers.
One note before we begin. This series is my own reorganization of what I have learned about AI red teaming over the past six months, combined with public frameworks, public research, and public labs, explained in my own words. If you want to learn this field in a more complete and systematic way, I still recommend going straight to professional material such as HTB Academy and OffSec to build a more solid foundation.
Why Is This Series Called “AI Dark Arts”?#
The first time people use generative AI, many of them feel like they are watching magic. You type in a prompt, and it writes code, organizes documents, translates articles, analyzes data, and even operates tools for you. Because of this, Chinese-speaking communities often call prompts “spells.”
A Chinese-language Facebook group named 「AI 咒語公社」, which translates roughly to “AI Spell Commune,” has more than 500,000 members.
But what if someone writes a bad spell on purpose?
- Make the AI leak information that was never meant to be public.
- Hide malicious instructions inside a document, an image, or a web page, and quietly change how the AI behaves.
- Trick the AI into calling a tool and performing an operation it should never perform.
- Steal information about the model, or feed the model incorrect or malicious data.
That is why I decided to call this series AI Dark Arts. Next, we will look at how these dark arts work, and at what we can do to defend against them.
Why Is AI Becoming a Target?#
AI is becoming a target mainly because today’s AI systems are usually connected to a large amount of valuable data and resources.
A single AI application may touch personal data, internal company documents, source code, account information, trade secrets, and conversation history. Conversation history is the part that ordinary users overlook most easily, because it can contain work content, internal information, and even sensitive material that nobody ever cleaned up.
On top of that, an AI application may hold permission to query databases, call APIs, send email, modify files, and even operate internal systems. For an attacker, the real attraction is what data the application can see, what permissions it holds, and what actions it can perform on behalf of a user.
AI is also moving very fast. In many systems, the security design, the permission model, and the defenses are still being worked out and adjusted. So an attacker only needs to find one weak link to obtain sensitive data, abuse a system feature, or extend their reach into the internal environment.
If we look at how generative AI works, many of these security problems come from its design. An LLM keeps predicting the next most likely token based on its training data and the current context, then joins those tokens into a complete answer.
It is very good at generating content, but that does not mean it understands what it just said. It also cannot always tell which parts of its context are system instructions, which parts are user input, and which parts are simply content from a document, a web page, or some other external source. Hallucination, prompt injection, and jailbreak are all tied to these characteristics.
A modern AI application is also a full architecture built from data, models, RAG, agents, external tools, APIs, and the systems around them. Every component you connect can add a new data flow, a new permission, and a new way in.
So when you analyze AI security, you cannot only stare at the chat window. Data, models, applications, tool permissions, systems, and the supply chain can all be part of the attack surface.
What Comes Next?#
I will spend many posts in this series discussing how to attack AI systems. What I hope readers take away, though, is the habit of treating AI as a system rather than as a single model or a chatbot.
Once you start looking at AI this way, you will notice questions that used to slip past you:
- Are the agent’s permissions too broad?
- Is the data behind RAG trustworthy?
- Could the external tools be abused?
- What attacks can the model itself face?
All of these belong to AI security.
I am still learning this field myself. I am organizing what I learn as I go, and sharing it with people who are interested in the same thing.
If this series gives you a better sense of the attack surface of AI systems, and a clearer picture of the risks they carry, that would be great.
In the next post, we will start with a problem that many people run into every day without knowing the reason behind it: why does AI make things up with such a straight face?