AnchorCrafter：通过人-物交互视频生成动画的网络锚销售您的产品

AnchorCrafter: Animate CyberAnchors Saling Your Products via Human-Object Interacting Video Generation

November 26, 2024

作者: Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xiaodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, Fan Tang

cs.AI

摘要

在线商务、广告和消费者参与方面，锚式产品推广视频的自动生成提供了很多机会。然而，尽管在姿势引导的人类视频生成方面取得了重大进展，但这仍然是一个具有挑战性的任务。为了解决这一挑战，我们确定将人-物互动（HOI）集成到姿势引导的人类视频生成中是一个核心问题。为此，我们引入了AnchorCrafter，这是一个基于扩散的新型系统，旨在生成具有目标人类和定制对象的2D视频，实现高视觉保真度和可控互动。具体而言，我们提出了两个关键创新：HOI-外观感知，可以增强对任意多视角透视图中对象外观的识别，并将对象和人类外观分离；以及HOI-运动注入，通过克服对象轨迹调节和相互遮挡管理方面的挑战，实现复杂的人-物互动。此外，我们引入了HOI-区域重新加权损失，这是一个训练目标，可以增强对对象细节的学习。大量实验证明，我们提出的系统在保留对象外观和形状意识方面优于现有方法，同时在保持人类外观和运动一致性方面也表现出色。项目页面：https://cangcz.github.io/Anchor-Crafter/

English

The automatic generation of anchor-style product promotion videos presents promising opportunities in online commerce, advertising, and consumer engagement. However, this remains a challenging task despite significant advancements in pose-guided human video generation. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Additionally, we introduce the HOI-region reweighting loss, a training objective that enhances the learning of object details. Extensive experiments demonstrate that our proposed system outperforms existing methods in preserving object appearance and shape awareness, while simultaneously maintaining consistency in human appearance and motion. Project page: https://cangcz.github.io/Anchor-Crafter/