DreamMix：在定制圖像修補中解耦對象屬性以增強可編輯性

DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting

November 26, 2024

作者: Yicheng Yang, Pengxiang Li, Lu Zhang, Liqian Ma, Ping Hu, Siyu Du, Yunzhi Zhuge, Xu Jia, Huchuan Lu

cs.AI

摘要

隨著擴散模型的最新進展，以及圖像編輯中主題驅動的圖像修補技術已成為一項熱門任務。先前的方法主要著重於保留身份特徵，但在維持插入物件的可編輯性方面遇到困難。為此，本文介紹了DreamMix，一種基於擴散的生成模型，擅長將目標物件插入到給定場景中的用戶指定位置，同時使其屬性可以進行任意文本驅動的修改。具體來說，我們利用先進的基礎修補模型，並引入了一個分離的局部-全局修補框架，以平衡精確的局部物件插入和有效的全局視覺連貫性。此外，我們提出了一個屬性解耦機制（ADM）和一個文本屬性替換（TAS）模組，分別用於改善基於文本的屬性引導的多樣性和區分能力。大量實驗表明，DreamMix 在各種應用場景中，包括物件插入、屬性編輯和小物件修補等方面，有效平衡了身份保留和屬性可編輯性。我們的程式碼公開在 https://github.com/mycfhs/DreamMix。

English

Subject-driven image inpainting has emerged as a popular task in image editing alongside recent advancements in diffusion models. Previous methods primarily focus on identity preservation but struggle to maintain the editability of inserted objects. In response, this paper introduces DreamMix, a diffusion-based generative model adept at inserting target objects into given scenes at user-specified locations while concurrently enabling arbitrary text-driven modifications to their attributes. In particular, we leverage advanced foundational inpainting models and introduce a disentangled local-global inpainting framework to balance precise local object insertion with effective global visual coherence. Additionally, we propose an Attribute Decoupling Mechanism (ADM) and a Textual Attribute Substitution (TAS) module to improve the diversity and discriminative capability of the text-based attribute guidance, respectively. Extensive experiments demonstrate that DreamMix effectively balances identity preservation and attribute editability across various application scenarios, including object insertion, attribute editing, and small object inpainting. Our code is publicly available at https://github.com/mycfhs/DreamMix.