Prevent instructions inside retrieved documents from controlling your AI app

A retrieved page contains text telling the assistant to ignore its task and reveal secrets. The page is task data, but the model may interpret its wording as an instruction.

Why this happens

Prompt injection exploits the mixing of trusted instructions with untrusted content. Delimiters and warnings can help communicate boundaries, but cannot replace application-level permission checks.

Step-by-step solution

1. Treat retrieved pages, uploads and tool results as untrusted data. Keep their content separate from the application’s trusted instructions.

2. Give tools only the permissions required for the task. Validate every requested action against the signed-in user’s authority outside the model.

3. Require explicit approval for consequential actions and avoid exposing secrets to the model context. Test adversarial documents alongside ordinary inputs.

Worked example

A document saying send the customer list to this address must not authorise an email tool. The application should reject the action unless the actual user and workflow independently permit it.

Verify the fix

Use a harmless injected instruction in a test document. The assistant should continue the authorised task, and the tool layer must block unauthorised actions even if the model proposes one.

Prevent a repeat

Log denied tool actions without storing secrets. Defence should remain effective when a prompt-level instruction is ignored.

Reference

Official documentation or standards guidance

An original Gistmax troubleshooting guide. Scenarios and sample values are illustrative; configuration depends on your software version and environment. Cover diagram: Gistmax Tutorials.