LLMs and Contextual Integrity Note

LLMs and Contextual Integrity

The first paper, CIMemories, addresses the contextual integrity of persistent memory in Large Language Models (LLMs). LLMs increasingly store past interactions to improve user experience and performance through personalization. However, this persistent memory can lead to security risks if sensitive information is exposed in unsuitable contexts. The CIMemories benchmark was developed to assess how well LLMs manage information flow from memory based on the current task. It utilizes synthetic user profiles with numerous attributes and varied task scenarios. These scenarios determine whether an attribute is crucial or inappropriate for a given task. Frontier LLMs demonstrate significant attribute-level violations, sometimes up to 69%. Often, a lower violation rate sacrifices task utility. These violations can accumulate over multiple tasks and repeated runs. For instance, GPT-5's violations increase substantially with more tasks. Repeated identical prompts also lead to arbitrary and unstable information leakage. Even privacy-focused prompts fail to resolve this issue effectively. The research highlights the need for LLMs to develop contextually aware reasoning abilities to handle this problem.