X-Dyna：表现力丰富的动态人类图像动画

摘要

我们介绍了一种名为X-Dyna的新型零样本、基于扩散的管道，用于通过来自驱动视频的面部表情和身体动作来为单个人类图像添加动画效果，生成逼真、具有上下文感知的主体和周围环境的动态。在以人体姿势控制为中心的先前方法基础上，X-Dyna解决了导致动态细节丢失的关键缺陷，增强了人类视频动画的逼真特性。我们方法的核心是动态适配器（Dynamics-Adapter），这是一个轻量级模块，能够有效地将参考外观背景整合到扩散骨干的空间注意力中，同时保留运动模块在合成流畅且复杂动态细节方面的能力。除了身体姿势控制，我们还将一个局部控制模块与我们的模型连接起来，以捕捉与身份解耦的面部表情，促进准确的表情转移，增强动画场景的逼真感。这些组件共同构成了一个统一框架，能够从各种人类和场景视频中学习人类运动和自然场景动态。全面的定性和定量评估表明，X-Dyna优于现有技术方法，创建了高度逼真和富有表现力的动画。代码可在https://github.com/bytedance/X-Dyna 上获得。

English

We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at https://github.com/bytedance/X-Dyna.

X-Dyna：表现力丰富的动态人类图像动画

X-Dyna: Expressive Dynamic Human Image Animation

摘要

Summary

Support

Support