草图：结构化知识增强文本理解以实现整体检索

SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval

December 19, 2024

作者: Aakash Mahalingam, Vinesh Kumar Gande, Aman Chadha, Vinija Jain, Divya Chaudhary

cs.AI

摘要

检索增强生成（RAG）系统已成为利用大规模语料库生成知情和与上下文相关响应的关键，显著减少大型语言模型中的幻觉。尽管取得了重大进展，但这些系统在高效处理和检索大型数据集的同时保持对上下文的全面理解方面仍存在困难。本文介绍了SKETCH，一种新颖的方法论，通过将语义文本检索与知识图谱相结合，增强了RAG检索过程，从而将结构化和非结构化数据融合为更全面的理解。SKETCH在检索性能方面表现出显著改进，并与传统方法相比保持了更优越的上下文完整性。在四个不同的数据集上进行评估：QuALITY、QASPER、NarrativeQA 和意大利烹饪，SKETCH在关键的RAGAS指标上（如答案相关性、忠实度、上下文精确度和上下文召回率）持续优于基准方法。值得注意的是，在意大利烹饪数据集上，SKETCH实现了0.94的答案相关性和0.99的上下文精确度，代表了所有评估指标中的最佳性能。这些结果突显了SKETCH在提供更准确和与上下文相关的响应方面的能力，为未来检索系统设立了新的基准。

English

Retrieval-Augmented Generation (RAG) systems have become pivotal in leveraging vast corpora to generate informed and contextually relevant responses, notably reducing hallucinations in Large Language Models. Despite significant advancements, these systems struggle to efficiently process and retrieve information from large datasets while maintaining a comprehensive understanding of the context. This paper introduces SKETCH, a novel methodology that enhances the RAG retrieval process by integrating semantic text retrieval with knowledge graphs, thereby merging structured and unstructured data for a more holistic comprehension. SKETCH, demonstrates substantial improvements in retrieval performance and maintains superior context integrity compared to traditional methods. Evaluated across four diverse datasets: QuALITY, QASPER, NarrativeQA, and Italian Cuisine-SKETCH consistently outperforms baseline approaches on key RAGAS metrics such as answer_relevancy, faithfulness, context_precision and context_recall. Notably, on the Italian Cuisine dataset, SKETCH achieved an answer relevancy of 0.94 and a context precision of 0.99, representing the highest performance across all evaluated metrics. These results highlight SKETCH's capability in delivering more accurate and contextually relevant responses, setting new benchmarks for future retrieval systems.