擴散模型的圖像複製偵測

Image Copy Detection for Diffusion Models

September 30, 2024

作者: Wenhao Wang, Yifan Sun, Zhentao Tan, Yi Yang

cs.AI

摘要

擴散模型生成的圖像在數字藝術和視覺營銷中越來越受歡迎。然而，這些生成的圖像可能複製現有內容，帶來內容原創性的挑戰。現有的圖像複製檢測（ICD）模型雖然在檢測手工複製品方面準確，但忽略了來自擴散模型的挑戰。這促使我們引入ICDiff，這是專門針對擴散模型的第一個ICD。為此，我們構建了一個擴散複製（D-Rep）數據集，並相應地提出了一種新穎的深度嵌入方法。D-Rep使用了一個最先進的擴散模型（穩定擴散 V1.5）生成了 40,000 張圖像複製對，這些對被手動標註為 6 個複製級別，範圍從 0（無複製）到 5（完全複製）。我們的方法 PDF-Embedding 將每個圖像複製對的複製級別轉換為概率密度函數（PDF）作為監督信號。直覺是相鄰複製級別的概率應該是連續且平滑的。實驗結果表明，PDF-Embedding 在 D-Rep 測試集上超越了協議驅動的方法和非 PDF 選擇。此外，通過利用 PDF-Embedding，我們發現知名擴散模型相對於開源畫廊的複製比例範圍從 10% 到 20%。

English

Images produced by diffusion models are increasingly popular in digital artwork and visual marketing. However, such generated images might replicate content from existing ones and pose the challenge of content originality. Existing Image Copy Detection (ICD) models, though accurate in detecting hand-crafted replicas, overlook the challenge from diffusion models. This motivates us to introduce ICDiff, the first ICD specialized for diffusion models. To this end, we construct a Diffusion-Replication (D-Rep) dataset and correspondingly propose a novel deep embedding method. D-Rep uses a state-of-the-art diffusion model (Stable Diffusion V1.5) to generate 40, 000 image-replica pairs, which are manually annotated into 6 replication levels ranging from 0 (no replication) to 5 (total replication). Our method, PDF-Embedding, transforms the replication level of each image-replica pair into a probability density function (PDF) as the supervision signal. The intuition is that the probability of neighboring replication levels should be continuous and smooth. Experimental results show that PDF-Embedding surpasses protocol-driven methods and non-PDF choices on the D-Rep test set. Moreover, by utilizing PDF-Embedding, we find that the replication ratios of well-known diffusion models against an open-source gallery range from 10% to 20%.