探究量化方法对大型语言模型安全性与可靠性的影响

Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models

February 18, 2025

作者: Artyom Kharinaev, Viktor Moskvoretskii, Egor Shvetsov, Kseniia Studenikina, Bykov Mikhail, Evgeny Burnaev

cs.AI

摘要

大型语言模型（LLMs）已成为应对现代挑战和实现实际应用的强大工具。然而，其高昂的计算成本仍是广泛采用的主要障碍。量化技术作为一种有前景的方法，旨在降低使用门槛并支持低资源设备的部署。尽管取得了这些进展，量化模型的安全性和可信度仍未得到充分探索，因为以往的研究往往忽视了当代架构，并依赖于过于简化的基准测试和评估方法。为填补这一空白，我们引入了OpenSafetyMini，一个新颖的开放式安全数据集，旨在更好地区分模型性能。我们使用四个基准测试（包括人工评估）对LLaMA和Mistral模型上的四种最先进的量化技术进行了评估。我们的研究结果表明，在4位精度下，最优的量化方法因模型而异，而在2位精度下，向量量化技术在安全性和可信度方面表现最佳，为未来研究奠定了基础。

English

Large Language Models (LLMs) have emerged as powerful tools for addressing modern challenges and enabling practical applications. However, their computational expense remains a significant barrier to widespread adoption. Quantization has emerged as a promising technique to democratize access and enable low resource device deployment. Despite these advancements, the safety and trustworthiness of quantized models remain underexplored, as prior studies often overlook contemporary architectures and rely on overly simplistic benchmarks and evaluations. To address this gap, we introduce OpenSafetyMini, a novel open-ended safety dataset designed to better distinguish between models. We evaluate 4 state-of-the-art quantization techniques across LLaMA and Mistral models using 4 benchmarks, including human evaluations. Our findings reveal that the optimal quantization method varies for 4-bit precision, while vector quantization techniques deliver the best safety and trustworthiness performance at 2-bit precision, providing foundation for future research.