Any2AnyTryon：利用自适应位置嵌入进行多功能虚拟服装任务

Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks

January 27, 2025

作者: Hailong Guo, Bohan Zeng, Yiren Song, Wentao Zhang, Chuang Zhang, Jiaming Liu

cs.AI

摘要

基于图像的虚拟试穿（VTON）旨在通过将输入服装转移到目标人物图像上生成虚拟试穿结果。然而，由于缺乏配对的服装-模特数据，现有方法很难在VTON中实现高泛化和质量。这也限制了生成无遮罩试穿的能力。为了解决数据稀缺问题，诸如稳定服装和MMTryon等方法采用了合成数据策略，有效增加了模特一侧的配对数据量。然而，现有方法通常局限于执行特定的试穿任务，并且缺乏用户友好性。为了增强VTON生成的泛化性和可控性，我们提出了Any2AnyTryon，可以根据不同的文本指令和模特服装图像生成试穿结果，以满足各种需求，消除了对口罩、姿势或其他条件的依赖。具体而言，我们首先构建了虚拟试穿数据集LAION-Garment，这是已知规模最大的开源服装试穿数据集。然后，我们引入自适应位置嵌入，使模型能够根据不同尺寸和类别的输入图像生成令人满意的穿戴模特图像或服装图像，从而显著提高了VTON生成的泛化性和可控性。在我们的实验中，我们展示了Any2AnyTryon的有效性，并将其与现有方法进行了比较。结果显示，Any2AnyTryon实现了灵活、可控和高质量的基于图像的虚拟试穿生成。

English

Image-based virtual try-on (VTON) aims to generate a virtual try-on result by transferring an input garment onto a target person's image. However, the scarcity of paired garment-model data makes it challenging for existing methods to achieve high generalization and quality in VTON. Also, it limits the ability to generate mask-free try-ons. To tackle the data scarcity problem, approaches such as Stable Garment and MMTryon use a synthetic data strategy, effectively increasing the amount of paired data on the model side. However, existing methods are typically limited to performing specific try-on tasks and lack user-friendliness. To enhance the generalization and controllability of VTON generation, we propose Any2AnyTryon, which can generate try-on results based on different textual instructions and model garment images to meet various needs, eliminating the reliance on masks, poses, or other conditions. Specifically, we first construct the virtual try-on dataset LAION-Garment, the largest known open-source garment try-on dataset. Then, we introduce adaptive position embedding, which enables the model to generate satisfactory outfitted model images or garment images based on input images of different sizes and categories, significantly enhancing the generalization and controllability of VTON generation. In our experiments, we demonstrate the effectiveness of our Any2AnyTryon and compare it with existing methods. The results show that Any2AnyTryon enables flexible, controllable, and high-quality image-based virtual try-on generation.https://logn-2024.github.io/Any2anyTryonProjectPage/