3DV-TON：基於擴散模型的紋理化3D引導一致性視頻試穿

3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models

April 24, 2025

作者: Min Wei, Chaohui Yu, Jingkai Zhou, Fan Wang

cs.AI

摘要

影片試衣技術旨在將影片中的服裝替換為目標衣物。現有方法在處理複雜的服裝圖案和多樣的體態姿勢時，難以生成高品質且時間上一致的結果。我們提出了3DV-TON，這是一個基於擴散模型的新框架，用於生成高保真且時間上一致的影片試衣效果。我們的方法採用生成的可動畫紋理3D網格作為顯式的幀級指導，從而緩解模型過於注重外觀保真度而犧牲動作連貫性的問題。這通過允許直接參考整個影片序列中一致的服裝紋理運動來實現。所提出的方法具有一個自適應的管道，用於生成動態3D指導：(1) 選擇一個關鍵幀進行初始的2D圖像試衣，隨後(2) 重建並動畫化一個與原始影片姿勢同步的紋理3D網格。我們進一步引入了一種穩健的矩形遮罩策略，成功減輕了在動態人體和服裝運動期間因服裝信息洩漏而導致的偽影傳播。為了推動影片試衣研究的發展，我們引入了HR-VVT，這是一個高解析度的基準數據集，包含130個影片，涵蓋多種服裝類型和場景。定量和定性結果顯示了我們相較於現有方法的優越性能。項目頁面鏈接如下：https://2y7c3.github.io/3DV-TON/

English

Video try-on replaces clothing in videos with target garments. Existing methods struggle to generate high-quality and temporally consistent results when handling complex clothing patterns and diverse body poses. We present 3DV-TON, a novel diffusion-based framework for generating high-fidelity and temporally consistent video try-on results. Our approach employs generated animatable textured 3D meshes as explicit frame-level guidance, alleviating the issue of models over-focusing on appearance fidelity at the expanse of motion coherence. This is achieved by enabling direct reference to consistent garment texture movements throughout video sequences. The proposed method features an adaptive pipeline for generating dynamic 3D guidance: (1) selecting a keyframe for initial 2D image try-on, followed by (2) reconstructing and animating a textured 3D mesh synchronized with original video poses. We further introduce a robust rectangular masking strategy that successfully mitigates artifact propagation caused by leaking clothing information during dynamic human and garment movements. To advance video try-on research, we introduce HR-VVT, a high-resolution benchmark dataset containing 130 videos with diverse clothing types and scenarios. Quantitative and qualitative results demonstrate our superior performance over existing methods. The project page is at this link https://2y7c3.github.io/3DV-TON/