从视频扩散模型中提取关节运动学信息

Articulated Kinematics Distillation from Video Diffusion Models

April 1, 2025

作者: Xuan Li, Qianli Ma, Tsung-Yi Lin, Yongxin Chen, Chenfanfu Jiang, Ming-Yu Liu, Donglai Xiang

cs.AI

摘要

我们提出了关节运动蒸馏（Articulated Kinematics Distillation, AKD）框架，该框架通过融合基于骨骼的动画与现代生成模型的优势，来生成高保真角色动画。AKD采用基于骨骼的表示方法处理装配好的3D资产，通过聚焦于关节层面的控制，显著降低了自由度（Degrees of Freedom, DoFs），从而实现高效、一致的运动合成。借助预训练视频扩散模型的分数蒸馏采样（Score Distillation Sampling, SDS），AKD在保持结构完整性的同时，蒸馏出复杂的关节运动，克服了4D神经变形场在维持形状一致性方面面临的挑战。此方法天然兼容基于物理的模拟，确保了物理上可信的交互。实验表明，在文本到4D生成任务上，AKD相较于现有工作，实现了更优的3D一致性与运动质量。项目页面：https://research.nvidia.com/labs/dir/akd/

English

We present Articulated Kinematics Distillation (AKD), a framework for generating high-fidelity character animations by merging the strengths of skeleton-based animation and modern generative models. AKD uses a skeleton-based representation for rigged 3D assets, drastically reducing the Degrees of Freedom (DoFs) by focusing on joint-level control, which allows for efficient, consistent motion synthesis. Through Score Distillation Sampling (SDS) with pre-trained video diffusion models, AKD distills complex, articulated motions while maintaining structural integrity, overcoming challenges faced by 4D neural deformation fields in preserving shape consistency. This approach is naturally compatible with physics-based simulation, ensuring physically plausible interactions. Experiments show that AKD achieves superior 3D consistency and motion quality compared with existing works on text-to-4D generation. Project page: https://research.nvidia.com/labs/dir/akd/