在大型语言和视觉-语言模型中学习适应性风险管理的符合性弃权策略

Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models

February 8, 2025

作者: Sina Tayebati, Divake Kumar, Nastaran Darabi, Dinithi Jayasuriya, Ranganath Krishnan, Amit Ranjan Trivedi

cs.AI

摘要

大型语言和视觉-语言模型（LLMs/VLMs）越来越多地应用于安全关键应用中，然而它们不透明的决策过程使风险评估和可靠性变得复杂。不确定性量化（UQ）有助于评估预测的置信度，并在不确定性较高时实现放弃。符合性预测（CP），作为一种主要的UQ方法，提供统计保证，但依赖于静态阈值，这些阈值无法适应任务复杂性和不断变化的数据分布，导致准确性、覆盖率和信息量之间的次优权衡。为了解决这个问题，我们提出了可学习的符合性放弃，将强化学习（RL）与CP相结合，以动态优化放弃阈值。通过将CP阈值视为自适应动作，我们的方法平衡了多个目标，最小化预测集大小的同时保持可靠的覆盖范围。在各种LLM/VLM基准测试中进行了广泛评估，结果显示我们的方法优于最不明确分类器（LAC）和自适应预测集（APS），将准确性提高了高达3.2％，将幻觉检测的AUROC提高了22.19％，将基于不确定性的选择性生成（AUARC）提高了21.17％，并将校准误差降低了70%-85%。这些改进在多个模型和数据集上都保持一致，同时始终满足90%的覆盖目标，确立了我们的方法作为在安全关键应用中进行可靠决策的更有效和灵活的解决方案。代码可在以下链接找到：{https://github.com/sinatayebati/vlm-uncertainty}。

English

Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability. Uncertainty quantification (UQ) helps assess prediction confidence and enables abstention when uncertainty is high. Conformal prediction (CP), a leading UQ method, provides statistical guarantees but relies on static thresholds, which fail to adapt to task complexity and evolving data distributions, leading to suboptimal trade-offs in accuracy, coverage, and informativeness. To address this, we propose learnable conformal abstention, integrating reinforcement learning (RL) with CP to optimize abstention thresholds dynamically. By treating CP thresholds as adaptive actions, our approach balances multiple objectives, minimizing prediction set size while maintaining reliable coverage. Extensive evaluations across diverse LLM/VLM benchmarks show our method outperforms Least Ambiguous Classifiers (LAC) and Adaptive Prediction Sets (APS), improving accuracy by up to 3.2%, boosting AUROC for hallucination detection by 22.19%, enhancing uncertainty-guided selective generation (AUARC) by 21.17%, and reducing calibration error by 70%-85%. These improvements hold across multiple models and datasets while consistently meeting the 90% coverage target, establishing our approach as a more effective and flexible solution for reliable decision-making in safety-critical applications. The code is available at: {https://github.com/sinatayebati/vlm-uncertainty}.