AnyMoLe：基于视频扩散模型的任意角色动作插值生成

AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models

March 11, 2025

作者: Kwan Yun, Seokhyeon Hong, Chaelin Kim, Junyong Noh

cs.AI

摘要

尽管基于学习的运动插值技术近期取得了进展，但一个关键限制却被忽视了：对角色特定数据集的依赖。在本研究中，我们提出了AnyMoLe，一种创新方法，通过利用视频扩散模型为任意角色生成运动插值帧，无需外部数据，从而解决了这一局限。我们的方法采用两阶段帧生成过程以增强上下文理解。此外，为了弥合现实世界与渲染角色动画之间的领域差距，我们引入了ICAdapt，一种针对视频扩散模型的微调技术。同时，我们提出了一种“运动-视频模仿”优化技术，使得利用2D和3D感知特征为具有任意关节结构的角色实现无缝运动生成成为可能。AnyMoLe显著降低了对数据的依赖，同时生成平滑且逼真的过渡，使其适用于广泛的运动插值任务。

English

Despite recent advancements in learning-based motion in-betweening, a key limitation has been overlooked: the requirement for character-specific datasets. In this work, we introduce AnyMoLe, a novel method that addresses this limitation by leveraging video diffusion models to generate motion in-between frames for arbitrary characters without external data. Our approach employs a two-stage frame generation process to enhance contextual understanding. Furthermore, to bridge the domain gap between real-world and rendered character animations, we introduce ICAdapt, a fine-tuning technique for video diffusion models. Additionally, we propose a ``motion-video mimicking'' optimization technique, enabling seamless motion generation for characters with arbitrary joint structures using 2D and 3D-aware features. AnyMoLe significantly reduces data dependency while generating smooth and realistic transitions, making it applicable to a wide range of motion in-betweening tasks.