SKETCH：結構化知識增強文本理解以進行整體檢索

SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval

December 19, 2024

作者: Aakash Mahalingam, Vinesh Kumar Gande, Aman Chadha, Vinija Jain, Divya Chaudhary

cs.AI

摘要

檢索增強生成（RAG）系統已成為利用龐大語料庫生成知情且與上下文相關回應的關鍵，顯著減少大型語言模型中的幻覺。儘管取得了重大進展，這些系統在有效處理和檢索大型數據集的同時，仍難以保持對上下文的全面理解。本文介紹了SKETCH，一種新穎的方法，通過將語義文本檢索與知識圖譜相結合，從而將結構化和非結構化數據融合，以實現更全面的理解。SKETCH展示了在檢索性能方面的顯著改進，並與傳統方法相比保持了優越的上下文完整性。在四個不同的數據集上進行評估：QuALITY、QASPER、NarrativeQA和意大利烹飪，SKETCH在關鍵的RAGAS指標（如答案相關性、忠實度、上下文精確度和上下文召回率）上持續優於基準方法。值得注意的是，在意大利烹飪數據集上，SKETCH實現了0.94的答案相關性和0.99的上下文精確度，代表了在所有評估指標中的最高性能。這些結果突顯了SKETCH在提供更準確和與上下文相關的回應方面的能力，為未來檢索系統設定了新的基準。

English

Retrieval-Augmented Generation (RAG) systems have become pivotal in leveraging vast corpora to generate informed and contextually relevant responses, notably reducing hallucinations in Large Language Models. Despite significant advancements, these systems struggle to efficiently process and retrieve information from large datasets while maintaining a comprehensive understanding of the context. This paper introduces SKETCH, a novel methodology that enhances the RAG retrieval process by integrating semantic text retrieval with knowledge graphs, thereby merging structured and unstructured data for a more holistic comprehension. SKETCH, demonstrates substantial improvements in retrieval performance and maintains superior context integrity compared to traditional methods. Evaluated across four diverse datasets: QuALITY, QASPER, NarrativeQA, and Italian Cuisine-SKETCH consistently outperforms baseline approaches on key RAGAS metrics such as answer_relevancy, faithfulness, context_precision and context_recall. Notably, on the Italian Cuisine dataset, SKETCH achieved an answer relevancy of 0.94 and a context precision of 0.99, representing the highest performance across all evaluated metrics. These results highlight SKETCH's capability in delivering more accurate and contextually relevant responses, setting new benchmarks for future retrieval systems.