Definition
Role confusion is the finding that LLMs infer the source/role of text (user, tool, assistant, chain-of-thought) primarily from writing style rather than architectural role tags — making style-spoofable privilege boundaries a structural vulnerability.
Key Points
- Core thesis of ICML 2026 paper “Prompt Injection as Role Confusion” (Ye, Cui, Hadfield-Menell) (2026-07-30-cot-forgery-arxiv)
- Enables cot-forgery and predicts success of standard prompt-injection
- Training-only defenses incomplete; layered runtime controls needed for sensitive systems (Florian Tramèr commentary via MIT TR)