AnchorCrafter：透過人物物件互動影片生成來展示您的產品

AnchorCrafter: Animate CyberAnchors Saling Your Products via Human-Object Interacting Video Generation

November 26, 2024

作者: Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xiaodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, Fan Tang

cs.AI

摘要

在線商務、廣告和消費者參與方面，自動生成錨式產品推廣視頻帶來了許多機遇。然而，儘管姿勢引導的人類視頻生成取得了重大進展，但這仍然是一項具有挑戰性的任務。為應對這一挑戰，我們確定將人物-物體交互（HOI）整合到姿勢引導的人類視頻生成中是一個核心問題。為此，我們引入了AnchorCrafter，這是一個基於擴散的新型系統，旨在生成具有目標人物和定制物體的2D視頻，實現高視覺保真度和可控交互。具體來說，我們提出了兩個關鍵創新：HOI-外觀感知，這有助於從任意多視角識別物體外觀並解開物體和人物外觀之間的關係，以及HOI-運動注入，通過克服物體軌跡條件和相互遮擋管理方面的挑戰，實現複雜的人物-物體交互作用。此外，我們引入了HOI-區域重新加權損失，這是一個訓練目標，有助於學習物體細節。大量實驗表明，我們提出的系統在保留物體外觀和形狀感知方面優於現有方法，同時在保持人物外觀和運動一致性方面也表現出色。項目頁面：https://cangcz.github.io/Anchor-Crafter/

English

The automatic generation of anchor-style product promotion videos presents promising opportunities in online commerce, advertising, and consumer engagement. However, this remains a challenging task despite significant advancements in pose-guided human video generation. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Additionally, we introduce the HOI-region reweighting loss, a training objective that enhances the learning of object details. Extensive experiments demonstrate that our proposed system outperforms existing methods in preserving object appearance and shape awareness, while simultaneously maintaining consistency in human appearance and motion. Project page: https://cangcz.github.io/Anchor-Crafter/