Exam-style
A penetration test against a subscription assistant returns four findings. First, a tampered knowledge-base article persuaded the model to start a refund. Second, a completion echoed a card number that had arrived in context. Third, the agent's action group could cancel any subscription, not only the caller's. Fourth, a model-generated string was rendered straight into an internal dashboard as HTML. Which single control closes the third finding?
Reveal the answer
D. Narrow the action group to a single cancel operation and check server-side that the authenticated caller owns the subscription named in the request
Each finding belongs to a different layer, and four controls that look interchangeable are not. The tampered article is indirect prompt injection, closed by the guardrail prompt-attack filter applied to retrieved content. The echoed card number is sensitive information disclosure, closed by the sensitive-information filter on the response. The rendered HTML is insecure output handling, closed by encoding at the rendering layer, which is a web control that happens to have a model upstream of it. The over-broad action group is excessive agency, and no filter on text can fix it: the model asked for something it was allowed to ask for, and the platform did it. Narrowing the tool schema to one operation and checking ownership server-side is the fix, because the authorisation decision has to live where the identity lives and not in the prompt. The system-prompt instruction is the tempting answer, since it reads like a rule; a model that has been talked into a refund can be talked past a sentence.
Q. A pen test finds a tampered article that triggered a refund, a leaked card number, an action group that can cancel anyone’s subscription, and model output rendered as HTML. Which one is not fixed by a filter?
A. The action group. That finding is excessive agency, so the fix is a narrower tool schema plus a server-side check that the authenticated caller owns the subscription named in the request. The other three are closed by a prompt-attack filter over retrieved content, a sensitive-information filter on the response, and output encoding at the rendering layer.
Why? Those four labels come from the OWASP Top 10 for LLM Applications, which is the taxonomy a security reviewer will write findings against: prompt injection, sensitive information disclosure, excessive agency, and insecure output handling (renamed improper output handling in the current list). Reading a report gets easier once each label points at a control that lives outside the model. Prompt injection and jailbreak detection mechanisms sit in front of the model on both the user turn and retrieved passages. Disclosure controls sit on the response. Output handling sits in whatever renders the string, and stays a plain web problem. Agency sits in the tool schema and the identity the tool runs as, which is why an over-broad action group survives every filter you add. Defense-in-depth safety systems get built this way, one layer per class of finding, and a red-team exercise you run yourself produces the same four kinds of finding before someone else’s report does.