利用相关反馈嵌入进行零样本密集检索

Zero-Shot Dense Retrieval with Embeddings from Relevance Feedback

October 28, 2024

作者: Nour Jedidi, Yung-Sung Chuang, Leslie Shing, James Glass

cs.AI

摘要

在没有相关监督的情况下构建有效的密集检索系统仍然很困难。最近的研究尝试通过使用大型语言模型（LLM）生成假设文档来克服这一挑战，以便找到最接近的真实文档。然而，这种方法仅依赖于LLM具有与查询相关的领域特定知识，这可能并不实际。此外，生成假设文档可能效率低下，因为它需要LLM为每个查询生成大量标记。为了解决这些挑战，我们引入了来自相关反馈的真实文档嵌入（ReDE-RF）。受相关反馈启发，ReDE-RF提议将假设文档生成重新构建为一个相关性估计任务，利用LLM选择应该用于最近邻搜索的文档。通过这种重新构建，LLM不再需要领域特定知识，而只需要判断什么是相关的。此外，相关性估计只需要LLM输出一个标记，从而提高了搜索延迟。我们的实验表明，ReDE-RF在一系列低资源检索数据集上始终优于最先进的零-shot密集检索方法，同时在每个查询的延迟方面也取得了显著改进。

English

Building effective dense retrieval systems remains difficult when relevance supervision is not available. Recent work has looked to overcome this challenge by using a Large Language Model (LLM) to generate hypothetical documents that can be used to find the closest real document. However, this approach relies solely on the LLM to have domain-specific knowledge relevant to the query, which may not be practical. Furthermore, generating hypothetical documents can be inefficient as it requires the LLM to generate a large number of tokens for each query. To address these challenges, we introduce Real Document Embeddings from Relevance Feedback (ReDE-RF). Inspired by relevance feedback, ReDE-RF proposes to re-frame hypothetical document generation as a relevance estimation task, using an LLM to select which documents should be used for nearest neighbor search. Through this re-framing, the LLM no longer needs domain-specific knowledge but only needs to judge what is relevant. Additionally, relevance estimation only requires the LLM to output a single token, thereby improving search latency. Our experiments show that ReDE-RF consistently surpasses state-of-the-art zero-shot dense retrieval methods across a wide range of low-resource retrieval datasets while also making significant improvements in latency per-query.