An AI application may encounter instructions inside a webpage, document or message it was asked to analyze. If the application treats that material as an instruction from its operator, the source has crossed a trust boundary. OWASP’s work on language-model application risks provides a starting point for examining this class of problem.
Treat outside material as evidence
A document can contain useful facts and hostile instructions at the same time. A robust workflow should preserve the distinction between the task defined by the user and the material retrieved to complete it. Stronger prompt wording may help express that distinction, but it cannot replace restrictions on the actions the application can perform.
Consider a hypothetical invoice assistant. Reading a supplier’s attachment should not grant permission to change payment details or send the company’s records to a new address. Those operations need independent validation and narrowly scoped capabilities that do not originate in the document being read.
Reduce the consequences of a mistake
Least-privilege access, controlled destinations and explicit approval for consequential changes limit what can happen if a model follows an inappropriate instruction. The system should also log attempted actions in a way that allows an operator to understand which information influenced the decision.
Testing should include documents that mix legitimate content with misleading requests. Teams can check whether the application continues its assigned task, exposes sensitive information or tries to use an unexpected tool. Successful behavior on ordinary documents is not sufficient evidence about these adversarial cases.
Make recovery part of the design
Operators need a way to revoke access, pause a workflow and inspect affected actions. Security becomes more practical when the team has rehearsed a bounded failure rather than assuming the right prompt will prevent every one.
