How do I defend a large language model application from attack?

LLM applications fail in ways traditional applications do not. The model treats every input it sees as instructions of equal authority, which means a malicious document or a poisoned search result can override the system prompt and exfiltrate data or misuse tools. The defense pattern starts from this assumption.

Layer one is input handling. Separate the user-supplied content from the system prompt with structural delimiters the model is trained to respect (XML tags, role boundaries, the Anthropic structured-prompt format). Validate length, content type, and provenance. Sanitize HTML when the model emits markup. Anthropic and OpenAI documentation cover the recommended boundary patterns.

Layer two is tool scoping. Every function the model can call should run under least privilege. A document-reader tool should not be able to write. A web-fetch tool should not be able to call internal services. Every tool should log invocations with the originating prompt context so an attack is reviewable after the fact.

Layer three is prompt-injection detection. Specialized models and rule engines can score whether an input contains likely injection patterns. Detection is imperfect but cuts the rate of successful attacks substantially. The OWASP LLM Top 10 (LLM01) lists the canonical patterns.

Layer four is rate limiting and quota enforcement. Most exfiltration attacks involve high-volume query patterns. Standard rate limits plus content-based anomaly detection (unusual token-count distributions, unusual tool-use ratios) catch most of these before significant data leaves.

Layer five is adversarial evaluation. Before launch, run an AI red team against the system using the MITRE ATLAS technique catalog. Treat the red-team output as a release-blocking checklist. Some failures will be acceptable risk; others will require architecture changes.

Layer six is incident response readiness. AI incident response shares vocabulary with traditional security incident response but adds prompts and model versions to the evidence chain. Run a tabletop exercise before launch.

These layers together are the practical floor in 2026. Skipping any layer creates predictable failure modes the threat-actor community already knows how to exploit.