解谜！隐秘的成员推断用于检索增强生成

摘要

检索增强生成（RAG）使大型语言模型（LLMs）能够利用外部知识数据库生成基于事实的响应，而无需改变模型参数。尽管权重调整的缺失防止了通过模型参数的泄漏，但它引入了推理对手利用模型上下文中检索文档的风险。现有的成员推理和数据提取方法通常依赖越狱或精心设计的不自然查询，这些方法很容易被检测或通过RAG系统中常见的查询重写技术挫败。在这项工作中，我们提出了询问攻击（IA），这是一种针对RAG数据存储中文档的成员推理技术。通过构建只有在目标文档存在时才能回答的自然文本查询，我们的方法展示了成功推理只需30个查询，同时保持隐蔽性；与现有方法生成的对抗提示相比，直接的检测器识别到我们的攻击生成的对抗提示的频率高出约76倍。我们观察到，在各种RAG配置中，与先前的推理攻击相比，TPR@1%FPR提高了2倍，同时每个文档推理的成本低于0.02美元。

English

Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to generate grounded responses by leveraging external knowledge databases without altering model parameters. Although the absence of weight tuning prevents leakage via model parameters, it introduces the risk of inference adversaries exploiting retrieved documents in the model's context. Existing methods for membership inference and data extraction often rely on jailbreaking or carefully crafted unnatural queries, which can be easily detected or thwarted with query rewriting techniques common in RAG systems. In this work, we present Interrogation Attack (IA), a membership inference technique targeting documents in the RAG datastore. By crafting natural-text queries that are answerable only with the target document's presence, our approach demonstrates successful inference with just 30 queries while remaining stealthy; straightforward detectors identify adversarial prompts from existing methods up to ~76x more frequently than those generated by our attack. We observe a 2x improvement in TPR@1%FPR over prior inference attacks across diverse RAG configurations, all while costing less than $0.02 per document inference.

解谜！隐秘的成员推断用于检索增强生成

Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation

摘要

Summary

Support