VLMGuard：透過未標記數據保護VLM免受惡意提示的攻擊

摘要

視覺語言模型（VLMs）對於上下文理解視覺和文本信息至關重要。然而，它們對敵對操縱輸入的脆弱性帶來重大風險，導致輸出受損，並引發對VLM集成應用可靠性的擔憂。因此，檢測這些惡意提示對於維護對VLM生成的信任至關重要。在開發保護提示分類器時面臨的一個主要挑戰是缺乏大量標記的良性和惡意數據。為解決這個問題，我們引入了VLMGuard，一種新型學習框架，利用野外未標記的用戶提示進行惡意提示檢測。這些未標記的提示在VLM部署在開放世界時自然生成，包含良性和惡意信息。為了利用這些未標記的數據，我們提出了一種自動惡意估計分數，以區分這些未標記混合中的良性和惡意樣本，從而實現在其之上訓練二元提示分類器。值得注意的是，我們的框架不需要額外的人工標註，為現實應用提供了強大的靈活性和實用性。廣泛的實驗表明，VLMGuard實現了優越的檢測結果，明顯優於最先進的方法。免責聲明：本文可能包含冒犯性示例；請慎重閱讀。

English

Vision-language models (VLMs) are essential for contextual understanding of both visual and textual information. However, their vulnerability to adversarially manipulated inputs presents significant risks, leading to compromised outputs and raising concerns about the reliability in VLM-integrated applications. Detecting these malicious prompts is thus crucial for maintaining trust in VLM generations. A major challenge in developing a safeguarding prompt classifier is the lack of a large amount of labeled benign and malicious data. To address the issue, we introduce VLMGuard, a novel learning framework that leverages the unlabeled user prompts in the wild for malicious prompt detection. These unlabeled prompts, which naturally arise when VLMs are deployed in the open world, consist of both benign and malicious information. To harness the unlabeled data, we present an automated maliciousness estimation score for distinguishing between benign and malicious samples within this unlabeled mixture, thereby enabling the training of a binary prompt classifier on top. Notably, our framework does not require extra human annotations, offering strong flexibility and practicality for real-world applications. Extensive experiment shows VLMGuard achieves superior detection results, significantly outperforming state-of-the-art methods. Disclaimer: This paper may contain offensive examples; reader discretion is advised.

VLMGuard：透過未標記數據保護VLM免受惡意提示的攻擊

VLMGuard: Defending VLMs against Malicious Prompts via Unlabeled Data

摘要

Summary

Support

Support