OmniPaint：通过解耦的插入-移除修复技术掌握面向对象的编辑

OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting

March 11, 2025

作者: Yongsheng Yu, Ziyun Zeng, Haitian Zheng, Jiebo Luo

cs.AI

摘要

基于扩散的生成模型已在面向对象的图像编辑领域引发革命，然而其在真实物体移除与插入中的应用仍受限于物理效应复杂交互及配对训练数据不足等挑战。本研究中，我们提出了OmniPaint，一个将物体移除与插入重新定义为相互依存过程而非孤立任务的统一框架。通过利用预训练的扩散先验，以及包含初始配对样本优化和后续大规模非配对CycleFlow精炼的渐进式训练流程，OmniPaint实现了精确的前景消除与无缝对象插入，同时忠实保留了场景几何与内在属性。此外，我们创新的CFD指标为上下文一致性与对象幻觉提供了无需参考的稳健评估，为高保真图像编辑设立了新基准。项目页面：https://yeates.github.io/OmniPaint-Page/

English

Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data. In this work, we introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes rather than isolated tasks. Leveraging a pre-trained diffusion prior along with a progressive training pipeline comprising initial paired sample optimization and subsequent large-scale unpaired refinement via CycleFlow, OmniPaint achieves precise foreground elimination and seamless object insertion while faithfully preserving scene geometry and intrinsic properties. Furthermore, our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination, establishing a new benchmark for high-fidelity image editing. Project page: https://yeates.github.io/OmniPaint-Page/