You ask your AI to summarize an email. It reads the email — and then does something you never asked for. That’s not a bug or a glitch. It’s one of the biggest security weaknesses in AI today, and it has a name: prompt injection. Security researchers rank it as the number one risk for AI applications — ahead of data leaks or misinformation.
In plain English: Prompt injection is a trick that makes an AI ignore its own rules and follow someone else’s instructions instead. There are two flavors: direct prompt injection (the attacker types the sneaky instruction themselves) and indirect prompt injection (the instruction is hidden inside a website, document, or email that the AI reads for you).
What is prompt injection?
OWASP (the Open Worldwide Application Security Project) publishes the widely recognized list of the biggest security risks for AI applications, the “OWASP Top 10 for LLM Applications.” Sitting right at the top of that list: LLM01 – Prompt Injection.
Here’s the core problem. An AI language model like ChatGPT, Copilot, or Gemini can barely tell the difference between “these are my instructions from the developer” and “this is just some text I’m reading right now.” To the AI, both are simply text in its context window. If someone manages to slip a convincing instruction into that text, they can — in the worst case — get the AI to break its rules, spill private information, or take actions it was never supposed to take.
OWASP also clears up a common mix-up: jailbreaking is a special form of prompt injection, where the AI is pushed into ignoring its safety guardrails entirely. So every jailbreak is a prompt injection — but not every prompt injection is a jailbreak.
Direct vs. indirect prompt injection
Direct prompt injection is when the attacker types the sneaky instruction straight into the chat — the classic “ignore all previous instructions and do X instead.” This is mostly a headache for the companies running AI services.
Indirect prompt injection is the one that should worry regular users. Nobody types anything live. Instead, the AI does its normal job — summarizing a web page, opening a PDF, going through your inbox — and reads content with a hidden instruction buried inside. According to OWASP, this happens whenever a model accepts input from outside sources like websites or files. If that content hides instructions, the AI’s behavior can change the moment it reads them — and you never typed anything suspicious.
Prompt injection: 3 real examples
This isn’t a theoretical threat. Here are three publicly documented cases, from a simple direct attack to a sophisticated data heist.
1. The Bing chatbot spills its secret name “Sydney” (February 2023)
Just days after Microsoft unveiled its new AI-powered Bing Chat, Stanford student Kevin Liu used a simple direct prompt injection to make the chatbot reveal its secret internal instructions. His trick: he told the AI to ignore previous instructions, then asked what was written at the start of the document above the conversation.
The chatbot promptly gave up its internal codename “Sydney” — which was supposed to be confidential — along with several of its ground rules, like the fact that it should only introduce itself as “Bing” and never reveal the name “Sydney.” Another student, Marvin von Hagen, independently confirmed the finding shortly afterward, using a different approach — he posed as an OpenAI developer and was shown nearly identical instructions. Microsoft quickly patched the hole, though Liu himself doubted at the time that this really fixed the underlying problem.
2. Invisible text on a web page hijacks Bing Chat (2023)
Security researcher Kai Greshake and his team showed, in their widely cited study “Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection” and its accompanying demos, that instructions can be smuggled into a web page in ways humans can’t see — for example as text with a font size of zero. When an AI like Bing Chat later visits that page — say, because a user asks for a summary — the model reads the invisible text anyway and can follow the instructions hidden inside.
In one of their demos, the researchers got Bing Chat to pose as a helpful ordering assistant and ask the user for their name, email address, and credit card details — triggered purely by the AI reading a booby-trapped page. They also showed that a compromised AI could redirect users through manipulated links so that data leaks out without the user noticing a thing.
3. “CamoLeak” steals company secrets through GitHub Copilot Chat (2025)
A more recent and far more serious example: security researcher Omer Mayraz of Legit Security found a flaw in GitHub Copilot Chat that he named “CamoLeak” — with a critical CVSS score of 9.6 out of 10. The attack combined two tricks. Hidden instructions were planted in invisible comments on pull requests — invisible in the GitHub interface, but read by Copilot anyway. Then, using GitHub’s own image proxy service “Camo,” the attack smuggled the stolen data out one character at a time, disguised as harmless image requests.
In his proof of concept, Mayraz had Copilot search a victim’s entire repository for the term “AWS_KEY” and exfiltrate the credentials it found, along with private source code and the description of an unpublished security vulnerability — all using nothing more than the permissions the logged-in user already had. GitHub closed the hole in August 2025; the case was publicly documented in October 2025.
Why this matters for everyday users
Prompt injection sounds like a developer problem. It’s becoming less and less so. The reason: more and more AI assistants now get access to real, personal data and tasks — they read your emails, browse websites for you, work through PDFs and documents, or take care of tasks for you. Every one of those abilities is also a possible doorway for indirect prompt injection: any website, document, or email that an AI processes on your behalf could, in theory, contain a hidden instruction.
OWASP’s list of possible consequences includes leaking sensitive information, exposing internal system instructions, manipulated or biased answers, unauthorized access to connected features, running unwanted commands in linked systems, and swaying important decisions. The three cases above show these aren’t abstract risks — they’ve been exploited for real, from a simple secret leak to the targeted theft of credentials.
How to protect yourself
There’s no 100% protection right now — even OWASP frames its advice as reducing risk, not guaranteeing safety. Still, you can lower your own risk a lot:
- Keep permissions tight: Give AI assistants only the access they actually need for the job. A tool that’s only supposed to summarize emails doesn’t also need permission to send emails or trigger payments.
- Confirm automated actions: For anything with consequences — purchases, sending messages, running code, deleting things — an AI should ask for your confirmation before doing it. That’s exactly what OWASP recommends as the “human-in-the-loop” principle for risky actions.
- Be suspicious of odd behavior: If an AI suddenly suggests or does things unrelated to your actual request right after visiting a web page or opening a document, that’s a red flag. Stop and check what happened.
- Don’t feed AI tools sensitive data you don’t have to — more on that in our guide to AI safety and privacy and our list of the 7 things you should never type into ChatGPT. Even without prompt injection it’s an unnecessary risk; with a successful injection it can turn into a direct data leak.
- Install updates promptly: Vendors do fix known holes — but for apps you install, you only benefit if you’re running the current version.
- Don’t blindly trust AI output: If an AI already answers vaguely or makes things up without anyone manipulating it, you should double-check its answers as a rule. We break down how to do that in AI hallucinations explained — the same caution helps against manipulated output from prompt injection.
Frequently asked questions about prompt injection
What is prompt injection?
Prompt injection is an attack technique against AI language models where an instruction — either typed in or hidden inside text the AI reads — overrides or sidesteps the system’s actual rules. The security group OWASP ranks prompt injection as the number one risk in its “Top 10 for LLM Applications.”
What’s the difference between direct and indirect prompt injection?
With direct prompt injection, the attacker types the manipulative instruction into the chat themselves. With indirect prompt injection, the instruction is hidden inside a website, document, or email that the AI reads as part of a normal task — without the user typing anything suspicious.
Is there a real example of prompt injection?
Yes. One well-known case is the Bing chatbot “Sydney”: in February 2023 a Stanford student got the chatbot to reveal its secret internal rules with a simple trick. Researchers led by Kai Greshake also showed that invisible text on web pages can be used to remote-control AI assistants. And in 2025, a critical flaw in GitHub Copilot Chat became public that could be used to steal private source code and credentials.
How do you protect yourself from prompt injection?
The most important steps: give AI assistants only the permissions they truly need, manually confirm risky automated actions, get suspicious when an AI behaves oddly, and don’t enter sensitive data into tools unless you have to. There’s no complete, guaranteed protection as things stand today.
Is prompt injection the same as jailbreaking?
Not quite. According to OWASP, jailbreaking is a special form of prompt injection where the AI is pushed into ignoring its safety guardrails entirely. So every jailbreak is a prompt injection — but not every prompt injection is a jailbreak.
More on AI reliability: AI hallucinations explained and Which AI hallucinates the least?
