何时发言，何时弃权：具有弃权功能的对比解码

When to Speak, When to Abstain: Contrastive Decoding with Abstention

December 17, 2024

作者: Hyuhng Joon Kim, Youna Kim, Sang-goo Lee, Taeuk Kim

cs.AI

摘要

大型语言模型（LLMs）通过利用预训练知识（即参数化知识）和外部知识（即上下文知识），在各种任务中展现出卓越的性能。虽然已经付出了大量努力来利用这两种形式的知识，但模型缺乏任何相关知识的情况仍然未被充分探讨。这种限制可能导致幻觉等问题，降低可靠性并在高风险应用中产生潜在风险。为了解决这些限制，本文将任务范围扩展到用户请求由于缺乏相关知识而无法满足的情况。为此，我们引入了一种名为对比解码与弃权（CDA）的无需训练的解码方法，使LLMs能够在有相关知识可用时生成响应，否则选择弃权。CDA评估了每个知识对于给定查询的相关性，自适应地确定哪些知识应优先考虑或完全忽略。在三个问答数据集上对四个LLMs进行的大量实验表明，CDA能够同时有效地执行准确的生成和弃权。这些发现突显了CDA拓宽LLMs适用范围的潜力，提高可靠性并保持用户信任。

English

Large Language Models (LLMs) demonstrate exceptional performance across diverse tasks by leveraging both pre-trained knowledge (i.e., parametric knowledge) and external knowledge (i.e., contextual knowledge). While substantial efforts have been made to leverage both forms of knowledge, scenarios in which the model lacks any relevant knowledge remain underexplored. Such limitations can result in issues like hallucination, causing reduced reliability and potential risks in high-stakes applications. To address such limitations, this paper extends the task scope to encompass cases where the user's request cannot be fulfilled due to the lack of relevant knowledge. To this end, we introduce Contrastive Decoding with Abstention (CDA), a training-free decoding method that empowers LLMs to generate responses when relevant knowledge is available and to abstain otherwise. CDA evaluates the relevance of each knowledge for a given query, adaptively determining which knowledge to prioritize or which to completely ignore. Extensive experiments with four LLMs on three question-answering datasets demonstrate that CDA can effectively perform accurate generation and abstention simultaneously. These findings highlight CDA's potential to broaden the applicability of LLMs, enhancing reliability and preserving user trust.