何時發言，何時棄權：具對比性解碼與棄權

When to Speak, When to Abstain: Contrastive Decoding with Abstention

December 17, 2024

作者: Hyuhng Joon Kim, Youna Kim, Sang-goo Lee, Taeuk Kim

cs.AI

摘要

大型語言模型（LLMs）通過利用預訓練知識（即參數化知識）和外部知識（即上下文知識），在各種任務上展現出卓越的表現。儘管已經付出了大量努力來利用這兩種形式的知識，但模型缺乏任何相關知識的情況仍然未被充分探討。這種限制可能導致幻覺等問題，從而降低可靠性並在高風險應用中帶來潛在風險。為了解決這些限制，本文將任務範圍擴大到包括由於缺乏相關知識而無法滿足用戶請求的情況。為此，我們引入了對比解碼與棄權（CDA），這是一種無需訓練的解碼方法，使LLMs能夠在有相關知識時生成回應，否則則棄權。CDA評估了每個知識對於給定查詢的相關性，自適應地確定要優先考慮哪些知識或完全忽略哪些知識。在三個問答數據集上對四個LLMs進行的大量實驗表明，CDA能夠同時有效執行準確的生成和棄權。這些發現突顯了CDA擴大LLMs應用範圍的潛力，提高可靠性並保持用戶信任。

English

Large Language Models (LLMs) demonstrate exceptional performance across diverse tasks by leveraging both pre-trained knowledge (i.e., parametric knowledge) and external knowledge (i.e., contextual knowledge). While substantial efforts have been made to leverage both forms of knowledge, scenarios in which the model lacks any relevant knowledge remain underexplored. Such limitations can result in issues like hallucination, causing reduced reliability and potential risks in high-stakes applications. To address such limitations, this paper extends the task scope to encompass cases where the user's request cannot be fulfilled due to the lack of relevant knowledge. To this end, we introduce Contrastive Decoding with Abstention (CDA), a training-free decoding method that empowers LLMs to generate responses when relevant knowledge is available and to abstain otherwise. CDA evaluates the relevance of each knowledge for a given query, adaptively determining which knowledge to prioritize or which to completely ignore. Extensive experiments with four LLMs on three question-answering datasets demonstrate that CDA can effectively perform accurate generation and abstention simultaneously. These findings highlight CDA's potential to broaden the applicability of LLMs, enhancing reliability and preserving user trust.