01Why AI systems are newly attackable

Think of how a calculator works: you press buttons, it does the math, and there's no way for a number you type to suddenly become a new instruction. Traditional software keeps commands and data in separate lanes. A language model blurs that line. It reads everything — your request, the developer's setup, a web page it fetched, an email it's summarizing — as one continuous stream of text. And here's the catch: the model has no built-in way to tell which text is a real instruction and which is just content it was asked to look at. So a sentence buried in a document it reads can quietly act like a command. That single fact — instructions and data share one channel — is what makes these systems a brand-new kind of attack surface.

• Traditional software separates commands from data; a language model mixes them in one text stream.
• The model can't reliably tell trusted instructions from untrusted content it's merely reading.
• So content the model reads — a page, a file, an email — can be crafted to behave like a command.

The security community keeps a running list of these weaknesses: the OWASP Top 10 for LLM Applications. Throughout this module we'll point back to it, and to the NIST AI Risk Management Framework, so you can place each idea in a recognized map rather than learning it in isolation.

02Prompt injection: smuggling in a command

Prompt injection is the headline AI attack, and it follows directly from section one. Because the model can't separate instructions from data, an attacker can smuggle an instruction into the text the model reads so the model follows them instead of doing its real job. There are two flavors. Direct injection is typed straight into the prompt by whoever is talking to the model. Indirect injection is sneakier: the malicious instruction hides inside untrusted content the model later fetches — a web page, a PDF, an email — so the user never typed anything suspicious at all. Try it below: flip the first switch to slip hostile text into a page the assistant is summarizing, then flip the second to turn on the defense.

• Direct injection is typed into the prompt; indirect injection hides in untrusted content the model fetches.
• The defense isn't a magic filter — it's a stance: treat retrieved content as data to summarize, never as instructions to obey.
• This is risk LLM01 in the OWASP Top 10 for LLM Applications — the single most-discussed AI weakness.

03Jailbreaks, data theft & the lethal trifecta

Prompt injection is about which instructions a model follows. Two more ideas build on it. A jailbreak targets the model's safety rules — coaxing it to ignore its own guardrails. And exfiltration is the goal that makes injection truly dangerous: getting private data out to somewhere the attacker controls. Put the pieces together and you get a memorable rule of thumb, the lethal trifecta (a term popularized by Simon Willison). Switch between the three tabs to see each leg.

04Agents raise the stakes

So far we've talked about a model that produces text. An AI agent goes further: it can act — send email, run code, browse the web, call other tools. That's useful, but it changes the math. When a plain chatbot is injected, the worst case is a bad sentence. When an agent is injected, the worst case is a real-world action: forwarding your files, making a purchase, deleting data. Two patterns capture the agent-specific danger. Tool misuse is when injected text steers an agent's legitimate tools toward harm — convincing your email agent to forward private files. Excessive agency is the deeper, structural problem: giving an agent more capability, access, or autonomy than its task actually needs, so that if it is hijacked, the blast radius is huge. An agent that can read untrusted web pages, reach your inbox, and send outbound requests has quietly assembled all three legs of the lethal trifecta in a single program.

• Agents act, not just talk — so a successful injection becomes a real action, not just bad text.
• Tool misuse: injected instructions turn an agent's legitimate tools (email, code, APIs) against the user.
• Excessive agency: over-provisioned access and autonomy enlarge the damage from any single hijack.
• OWASP flags both excessive agency and insecure tool design as top LLM-application risks.

05The defenses that hold

You can't perfectly prevent prompt injection — so the goal shifts from "block every attack" to "make sure no single failure leads to harm." That's defense in depth: several independent controls, so an attacker has to beat all of them. The foundation is a mindset — never treat untrusted content as instructions; it is data to be processed, full stop. On top of that sit a handful of practical, widely-recommended controls.

• Never trust untrusted content as commands — treat retrieved pages, files, and emails strictly as data.
• Least privilege — give the model or agent only the access and tools its task truly requires.
• Input & output filtering — screen what goes in and validate what comes out before it's used.
• Human-in-the-loop — require a person to approve high-impact actions (sending money, deleting data).
• Sandboxing — isolate tool execution and limit outbound network access to cut exfiltration channels.
• Map to frameworks — the OWASP Top 10 for LLM Applications names the risks and mitigations; the NIST AI RMF structures how you govern, map, measure, and manage them over time; MITRE ATLAS catalogues real attacker techniques so you can reason about coverage.

06Check your understanding

07One important caveat, then go deeper