Indirect Prompt Injection Atta... Note

Indirect Prompt Injection Attacks Against LLM Assistants

The research paper "Invitation Is All You Need!" explores practical and dangerous promptware attacks against LLM-powered assistants. Promptware refers to malicious prompts designed to compromise the security of applications utilizing LLMs. The study investigates the risks posed by promptware to users of Gemini-powered assistants, including web, mobile, and Google Assistant. A novel Threat Analysis and Risk Assessment (TARA) framework was developed to evaluate these risks for end users. The research introduces Targeted Promptware Attacks, a new variant that uses indirect prompt injection through common user interactions like emails and calendar invitations. Fourteen attack scenarios were demonstrated across five threat classes: short-term context poisoning, permanent memory poisoning, tool misuse, automatic agent invocation, and automatic app invocation. These attacks can lead to various digital and physical consequences such as spam, phishing, disinformation, data exfiltration, and control of home automation devices. The paper reveals promptware's capability for on-device lateral movement, allowing malicious actions beyond the LLM application. The TARA framework indicated that 73% of analyzed threats posed a High-Critical risk to end users. The researchers discussed mitigations, demonstrating that these could reduce risks to Very Low-Medium. Google has deployed dedicated mitigations based on the findings disclosed by the researchers.