文本和视觉模态下的字符去学习

摘要

机器遗忘（MU）对于增强深度学习模型的隐私和安全性至关重要，尤其是对于大型多模态语言模型（MLLMs），通过消除特定的私人或危险信息。虽然在文本和视觉模态中，MU已经取得了显著进展，但多模态遗忘（MMU）仍然受到极大的忽视，部分原因是缺乏适用的开源基准。为了解决这个问题，我们引入了CLEAR，一个新的基准，旨在评估MMU方法。CLEAR包含200个虚构个体和3,700张图像，与相应的问答对相关联，使得能够在各种模态下进行彻底评估。我们评估了10种MU方法，并对其进行了适应以用于MMU，并突出了多模态遗忘特有的新挑战。我们还证明了简单的ell_1正则化对LoRA权重可以显著减轻灾难性遗忘，保持模型对保留数据的性能。该数据集可在https://huggingface.co/datasets/therem/CLEAR 上获得。

English

Machine Unlearning (MU) is critical for enhancing privacy and security in deep learning models, particularly in large multimodal language models (MLLMs), by removing specific private or hazardous information. While MU has made significant progress in textual and visual modalities, multimodal unlearning (MMU) remains significantly underexplored, partially due to the absence of a suitable open-source benchmark. To address this, we introduce CLEAR, a new benchmark designed to evaluate MMU methods. CLEAR contains 200 fictitious individuals and 3,700 images linked with corresponding question-answer pairs, enabling a thorough evaluation across modalities. We assess 10 MU methods, adapting them for MMU, and highlight new challenges specific to multimodal forgetting. We also demonstrate that simple ell_1 regularization on LoRA weights significantly mitigates catastrophic forgetting, preserving model performance on retained data. The dataset is available at https://huggingface.co/datasets/therem/CLEAR

文本和视觉模态下的字符去学习

CLEAR: Character Unlearning in Textual and Visual Modalities

摘要

Summary

Support

Support