↓ Skip to main content
Vivian’s Blog
  1. Posts/

Attacking AI Is More Than Prompt Injection｜My HTB Certified Offensive AI Expert (COAE) Experience

·976 words·5 mins
Table of Contents

Before this, I used generative AI the same way most people do. I used chatbots like Claude, ChatGPT, and Gemini to help with my work and learning. Earlier this year, I decided to find a course to learn how to attack AI systems. This field only became popular in the last few years, so there are not many complete courses about it. I have always liked how Hack The Box Academy builds its courses, and HTB happened to have one, so I chose the AI Red Teamer Path. When I was about halfway through the course, HTB released the Certified Offensive AI Expert (COAE) certification. So I decided to go for this certification.

Introduction｜What Does HTB Certified Offensive AI Expert (COAE) Teach?
#

Let me give a short introduction to this certification first, since it only launched this April and not many people know about it yet.

HTB Certified Offensive AI Expert is a hands-on AI Red Teaming certification that Hack The Box released this year. Unlike many courses that only focus on Prompt Injection, it starts from the basics of AI. It teaches you how to analyze and evaluate the security of AI systems, and how to use these weaknesses to attack them. The content covers basic algorithms and attacks on many different related applications. If you want to know more about the course content, I also wrote a learning log about the AI Red Teamer Path before. Feel free to read it if you are interested.

Exam Experience｜Coding Is Hard…
#

COAE is a fully hands-on certification. The whole exam environment has one host and three domains, with 7 flags in total. You need at least 6 flags to pass. The exam lasts 7 days. There is no live proctoring, but the time to write your report is also inside these 7 days. You must submit a full technical report before time runs out.

Even though the topic is AI Red Teaming, the exam is not just about talking with AI. During the exam, there are also some simple web penetration testing tasks. So if you already have a web pentest background, it will be easier to reach the AI systems quickly.

Python is also a must-have skill. Many parts of the course use Python examples to explain ideas, and you also need to write code during the exam. This skill matters a lot, because it decides whether you can get enough flags to pass. When you submit your report, you also need to include your full source code.

AI algorithms are closely tied to code, so I think having basic Python skills will lower the learning barrier a lot. This is especially true in the second half of the course, where you need to read and modify Python code. (I am not very good at coding, and this part was the hardest for me.) I also suggest saving all the example code while you study. It will help a lot during the exam.

As for the exam content, I felt that the course material matches the exam very well. If you finish every module carefully and understand the reasoning behind it, the exam should not feel too different from what you learned.

I think this certification fits people who already have some Python and basic web pentest skills. My advice is to build a solid foundation first before taking on this certification. This will make the learning process much smoother.

What I Gained｜Learning to Attack AI Also Made Me Better at Using AI
#

After finishing the whole learning path, I learned many attack techniques for AI systems from the course material. Before this, whenever I used AI, I only thought like a user. I only cared about which model was good for which task, and whether the answers were good. I never learned how things actually worked behind the scenes.

After I started studying, I realized that the chatbots and models I used before were only one small part of the whole AI system. Behind these applications, there are also data sources, RAG, Agents, Tool Calling, external APIs, and many different data flows. In many cases, the real attack surface is the permission design problem in how these components interact with each other. This is the biggest change in how I think after finishing the course. Now I think of AI as a whole system, and this way of thinking also changed how I build Skills. I started splitting different roles, such as management, execution, auditing, and feedback, so each role can work together instead of giving all the work to one single Agent. I also build a distilled knowledge base based on what each Skill needs.

Conclusion
#

AI moves forward very fast. New technologies and new ways to use them keep showing up all the time. Instead of always chasing the newest technology, I think it is more important to first understand the basic logic of how AI works. Once you have a solid foundation, you can understand new technologies much faster.

For me, the biggest thing I gained from HTB COAE is a correct way to think about AI. Even as things evolve from Prompt Engineer, Context Engineer, Harness Engineer, to Loop Engineer, I can quickly understand the logic behind each step, and this also helps me produce better work.

If you are also interested in AI, I think the HTB AI Red Teamer Path is a learning path worth your time. It explains how LLMs work starting from the basic concepts, and it also helps you understand AI Cybersecurity more deeply.