PhotoDoodle：从少量成对数据中学习艺术图像编辑

PhotoDoodle: Learning Artistic Image Editing from Few-Shot Pairwise Data

February 20, 2025

作者: Shijie Huang, Yiren Song, Yuxuan Zhang, Hailong Guo, Xueyin Wang, Mike Zheng Shou, Jiaming Liu

cs.AI

摘要

我们推出PhotoDoodle，一个创新的图像编辑框架，旨在通过让艺术家能够在照片上叠加装饰元素来促进照片涂鸦创作。照片涂鸦具有挑战性，因为插入的元素必须与背景无缝融合，这需要逼真的混合、透视对齐和上下文一致性。此外，背景必须保持原样不受扭曲，同时艺术家的独特风格需从有限的训练数据中高效捕捉。这些需求是先前主要关注全局风格迁移或区域修复的方法所未解决的。所提出的方法PhotoDoodle采用了两阶段训练策略。首先，我们利用大规模数据训练一个通用图像编辑模型OmniEditor。随后，我们使用EditLoRA通过一个小型、由艺术家精选的前后图像对数据集对该模型进行微调，以捕捉独特的编辑风格和技术。为了增强生成结果的一致性，我们引入了位置编码重用机制。此外，我们发布了一个包含六种高质量风格的PhotoDoodle数据集。大量实验证明，我们的方法在定制化图像编辑中展现出卓越的性能和鲁棒性，为艺术创作开辟了新的可能性。

English

We introduce PhotoDoodle, a novel image editing framework designed to facilitate photo doodling by enabling artists to overlay decorative elements onto photographs. Photo doodling is challenging because the inserted elements must appear seamlessly integrated with the background, requiring realistic blending, perspective alignment, and contextual coherence. Additionally, the background must be preserved without distortion, and the artist's unique style must be captured efficiently from limited training data. These requirements are not addressed by previous methods that primarily focus on global style transfer or regional inpainting. The proposed method, PhotoDoodle, employs a two-stage training strategy. Initially, we train a general-purpose image editing model, OmniEditor, using large-scale data. Subsequently, we fine-tune this model with EditLoRA using a small, artist-curated dataset of before-and-after image pairs to capture distinct editing styles and techniques. To enhance consistency in the generated results, we introduce a positional encoding reuse mechanism. Additionally, we release a PhotoDoodle dataset featuring six high-quality styles. Extensive experiments demonstrate the advanced performance and robustness of our method in customized image editing, opening new possibilities for artistic creation.