AI security
Prompt injection explained, for defenders
What is prompt injection? How hidden instructions in emails, files or web pages can steer AI assistants, and defences ASD, Microsoft and OWASP describe.
How to protect against prompt injection: what official guidance lists
Microsoft says reliably detecting indirect prompt injection is still an open research challenge, so official guidance focuses on limiting what an injected instruction could do.
- Give AI tools only the access they needThe ASD describes least privilege, OWASP describes limiting AI extensions' permissions to the minimum necessary, and Microsoft points to fine-grained permissions.Least privilege
- Require a person to approve high-impact actionsOWASP describes human-in-the-loop control for high-impact actions. Microsoft's example is that Copilot in Outlook leaves the user to approve and send the email.Human approval
- Log and monitor what goes in and comes outThe ASD describes logging inputs and outputs and establishing a baseline of the AI system's activity, so anomalies are easier to spot.Logging and monitoring
- Include AI systems in your incident response planThe ASD describes an incident response plan that accounts for AI systems, with vendor and customer responsibilities set out in the contract.Incident response and vendors
- Ask vendors how they've built and tested their AIThe ASD describes seeking vendors that are transparent about how they have developed and tested their AI systems.Questions from the ASD
Based on the ASD's 'Engaging with artificial intelligence', Microsoft Security Response Center's guidance on indirect prompt injection and OWASP's Top 10 for LLM applications, checked October 2026.
Key points
- The ASD describes prompt injection as an input manipulation attack that attempts to insert malicious instructions or hidden commands into an AI system.
- Microsoft describes indirect prompt injection, where instructions arrive through content such as emails, documents or web pages, as an inherent risk of modern large language models.
- OWASP says the impact of excessive agency depends on which systems the AI application can reach, which is why permissions matter as much as detection.
Prompt injection is an attack that tries to slip instructions into what an AI system reads, such as an email, file or web page, so that it does something its user or owner didn’t intend. It matters because AI assistants are increasingly able to read email, search files, browse websites and take actions for their users. This guide explains what prompt injection is, the difference between direct and indirect attacks, and the defences described by the Australian Signals Directorate (ASD), Microsoft and OWASP.
What prompt injection is
The ASD’s guidance Engaging with artificial intelligence describes prompt injection as “an input manipulation attack that attempts to insert malicious instructions or hidden commands” into an AI system.
OWASP, through its Gen AI Security Project, lists prompt injection as the first entry (LLM01) in its Top 10 for large language model (LLM) and generative AI applications. OWASP describes the list as “a global, community-driven effort” to identify and raise awareness of major security risks for these systems.
Direct and indirect prompt injection
Microsoft’s Security Response Center explains the difference:
- Direct prompt injection is when the attacker is the person using the AI tool.
- Indirect prompt injection involves “the attacker injecting instructions into the interaction between a victim user and the LLM.” Microsoft says the attacker’s text can reach the model through external sources such as web pages, emails, shared documents or tool outputs, and that “the prompt injection could be contained in an email sent by the attacker to the victim.”
Microsoft calls indirect prompt injection “an inherent risk” of modern LLMs. In the indirect case, the person affected is a legitimate user who did nothing wrong.
Why it matters more when AI can act
OWASP’s Top 10 also includes “excessive agency” (LLM06). OWASP defines it as “the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous” LLM outputs, and says its root cause is typically excessive functionality, excessive permissions or excessive autonomy. It lists “direct/indirect prompt injection from a malicious user” as a common trigger, and says the potential impact depends on which systems the AI application can reach.
How defenders limit prompt injection
Microsoft says “deterministically detecting indirect prompt injection is still an open research challenge”, and that its approach “does not rely on our ability to block all prompt injections.” The sources describe several layers.
Least privilege
The ASD describes granting privileges “based on the need-to-know principle and the principle of least privilege.” OWASP describes limiting the permissions AI extensions have to other systems “to the minimum necessary”. Microsoft says the risk can be “deterministically mitigated using fine-grained permissions and access controls.” In Microsoft 365, see admin roles and least privilege and app consent settings. For AI tools that act on their own, AI agents and permissions covers this in more depth, with a fictional example.
Human approval for high-impact actions
OWASP describes using “human-in-the-loop control to require a human to approve high-impact actions before they are taken.” Microsoft gives the example that, with Copilot in Outlook, “the user must explicitly approve the generated text and send the email themselves.”
Separating instructions from untrusted content, and detection
Microsoft describes hardened system prompts and a technique it calls Spotlighting, which helps the model separate instructions from untrusted text. It notes these preventive measures are probabilistic. Microsoft also describes a classifier, Prompt Shields, that flags injection attempts.
Logging and monitoring
The ASD describes logging and monitoring both inputs to and outputs from the AI system, and establishing “a baseline of the AI system’s activity” so anomalies are easier to spot. See audit logs and alerts in Microsoft 365.
Incident response and vendors
The ASD describes making sure the incident response plan “accounts for issues arising from, or to, its AI systems”, and that vendor and customer responsibilities for incident management are clearly defined in the contract. For a starting point, see the one-page cyber incident response plan template. The ASD also describes seeking vendors “that are transparent about how they have developed and tested their AI systems.”
Questions to ask before adopting an AI system
The ASD’s Engaging with artificial intelligence frames its mitigations as questions for organisations, including:
- “Does your organisation enforce multi-factor authentication?”
- “How will your organisation manage privileged access to the AI system?”
- “Can your organisation implement a trial of the AI system?”
- “Is the AI system secure-by-design, including its supply chain?”
- “Does your organisation understand the limits and constraints of the AI system?”
- “Does your organisation enforce logging and monitoring?”
- “What will your organisation do if something goes wrong with the AI system?”
Things to watch for
- AI tools that read outside content and can also act, such as an assistant that reads inbound email and can send messages or change files.
- Actions that happen without anyone approving them, especially sending, deleting or sharing.
- Broad access granted to get a pilot working, then never reduced.
- Logs that record only the chat, not the inputs the AI read or the actions it took.
Common questions
What is prompt injection?
It’s an attack that tries to insert instructions into an AI system so it does something its user or owner didn’t intend. The ASD describes it as “an input manipulation attack that attempts to insert malicious instructions or hidden commands” into an AI system. OWASP lists prompt injection as the first entry, LLM01, in its Top 10 for large language model and generative AI applications. See what prompt injection is.
What’s the difference between direct and indirect prompt injection?
In direct prompt injection, the attacker is the person using the AI tool. In indirect prompt injection, the instructions arrive through content the AI reads, such as a web page, email, shared document or tool output, and the person affected is a legitimate user who did nothing wrong. Microsoft calls indirect prompt injection an inherent risk of modern large language models. See direct and indirect prompt injection.
Can an email make my AI assistant do something I didn’t ask for?
That is the risk indirect prompt injection describes. Microsoft says the instructions “could be contained in an email sent by the attacker to the victim.” How much harm follows depends on what the assistant can do: OWASP says the impact depends on which systems the AI application can reach. Microsoft’s example is that, with Copilot in Outlook, the user must explicitly approve the generated text and send the email themselves. See why it matters more when AI can act.
Can prompt injection be prevented completely?
Not reliably, according to Microsoft. It says deterministically detecting indirect prompt injection is still an open research challenge, and that its approach doesn’t rely on blocking all prompt injections. Its defences combine probabilistic prevention, such as hardened system prompts, Spotlighting and the Prompt Shields classifier, with fine-grained permissions and access controls that limit what an AI system can do. See how defenders limit prompt injection.
How do I protect my organisation’s AI tools from prompt injection?
Official guidance focuses on limiting what an injected instruction could do. The ASD describes least privilege, and OWASP describes limiting AI extensions’ permissions to the minimum necessary and requiring a person to approve high-impact actions. The ASD also describes logging inputs and outputs, including AI systems in the incident response plan, and seeking vendors that are transparent about how they built and tested their AI. See least privilege and the following sections.
What do I do if I think an AI tool has been manipulated by prompt injection?
If an AI tool or connected account may have been misused, the ACSC takes reports through ReportCyber and runs a 24/7 hotline on 1300 CYBER1 (1300 292 371). The ASD describes an incident response plan that accounts for issues arising from AI systems, with vendor and customer responsibilities set out in the contract. Logs of the AI system’s inputs and outputs help show what it read and did. See incident response and vendors.
Where to get official help
- The ASD’s Engaging with artificial intelligence is the main Australian government reference.
- If an AI tool or connected account may have been misused, the ACSC takes reports through ReportCyber and runs a 24/7 hotline on 1300 CYBER1 (1300 292 371).