AnimateAnything：用於影片生成的一致且可控動畫

AnimateAnything: Consistent and Controllable Animation for Video Generation

November 16, 2024

作者: Guojun Lei, Chi Wang, Hong Li, Rong Zhang, Yikai Wang, Weiwei Xu

cs.AI

摘要

我們提出了一種統一的可控影片生成方法 AnimateAnything，有助於在各種情況下實現精確且一致的影片操作，包括攝影機軌跡、文字提示和使用者動作標註。具體而言，我們精心設計了一個多尺度控制特徵融合網絡，用於構建不同情況下的共同運動表示。它明確地將所有控制信息轉換為逐幀光流。然後，我們將光流作為運動先驗，引導最終的影片生成。此外，為了減少大範圍運動引起的閃爍問題，我們提出了一個基於頻率的穩定模塊。它可以通過確保影片的頻率域一致性來增強時間上的連貫性。實驗表明，我們的方法優於最先進的方法。有關更多細節和影片，請參閱網頁：https://yu-shaonian.github.io/Animate_Anything/。

English

We present a unified controllable video generation approach AnimateAnything that facilitates precise and consistent video manipulation across various conditions, including camera trajectories, text prompts, and user motion annotations. Specifically, we carefully design a multi-scale control feature fusion network to construct a common motion representation for different conditions. It explicitly converts all control information into frame-by-frame optical flows. Then we incorporate the optical flows as motion priors to guide final video generation. In addition, to reduce the flickering issues caused by large-scale motion, we propose a frequency-based stabilization module. It can enhance temporal coherence by ensuring the video's frequency domain consistency. Experiments demonstrate that our method outperforms the state-of-the-art approaches. For more details and videos, please refer to the webpage: https://yu-shaonian.github.io/Animate_Anything/.