DZone.com
Follow
The AI Memory Security Blueprint
Frontier large language models can be significantly manipulated through RAG systems. Injecting just five malicious documents into a large corpus allowed researchers to control LLM output nearly all the time. This attack, known as PoisonedRAG, bypasses direct model or retriever code access. Attackers simply introduce a poisoned document that gets indexed. Subsequent research, CorruptRAG, demonstrated that a single poisoned document could achieve a high success rate. This makes the threat more realistic as infecting a few documents is more plausible than many. Even a small percentage, like 0.04% of a corpus being poisoned, can lead to over 98% attack success. These poisoning attacks can cause system failures in a substantial portion of instances. The findings highlight that simply adding RAG for accuracy is not a risk-free endeavor. Robust security measures are crucial for enterprise RAG systems.