MotionCLR：通過理解注意機制實現動作生成和無需訓練的編輯

MotionCLR: Motion Generation and Training-free Editing via Understanding Attention Mechanisms

October 24, 2024

作者: Ling-Hao Chen, Wenxun Dai, Xuan Ju, Shunlin Lu, Lei Zhang

cs.AI

摘要

本研究探討人體動作生成的互動式編輯問題。先前的動作擴散模型缺乏對詞級文本-動作對應的明確建模和良好的可解釋性，因此限制了其精細編輯能力。為解決此問題，我們提出了一種基於注意力的動作擴散模型，即MotionCLR，具有清晰建模注意力機制。從技術上講，MotionCLR通過自注意力和交叉注意力分別對模態內和跨模態交互進行建模。具體而言，自注意力機制旨在衡量幀之間的序列相似性並影響動作特徵的順序。相比之下，交叉注意力機制旨在找到細粒度的詞序列對應並激活動作序列中相應的時間步。基於這些關鍵特性，我們通過操縱注意力映射開發了一套多功能的簡單而有效的動作編輯方法，例如動作（去）強調、原地動作替換和基於示例的動作生成等。為進一步驗證注意力機制的可解釋性，我們另外探索了通過注意力映射的動作計數和基於基準的動作生成能力的潛力。我們的實驗結果顯示，我們的方法在生成和編輯能力方面表現良好並具有良好的可解釋性。

English

This research delves into the problem of interactive editing of human motion generation. Previous motion diffusion models lack explicit modeling of the word-level text-motion correspondence and good explainability, hence restricting their fine-grained editing ability. To address this issue, we propose an attention-based motion diffusion model, namely MotionCLR, with CLeaR modeling of attention mechanisms. Technically, MotionCLR models the in-modality and cross-modality interactions with self-attention and cross-attention, respectively. More specifically, the self-attention mechanism aims to measure the sequential similarity between frames and impacts the order of motion features. By contrast, the cross-attention mechanism works to find the fine-grained word-sequence correspondence and activate the corresponding timesteps in the motion sequence. Based on these key properties, we develop a versatile set of simple yet effective motion editing methods via manipulating attention maps, such as motion (de-)emphasizing, in-place motion replacement, and example-based motion generation, etc. For further verification of the explainability of the attention mechanism, we additionally explore the potential of action-counting and grounded motion generation ability via attention maps. Our experimental results show that our method enjoys good generation and editing ability with good explainability.