MagicFace：使用動作單元控制進行高保真面部表情編輯

MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control

January 4, 2025

作者: Mengting Wei, Tuomas Varanka, Xingxun Jiang, Huai-Qian Khor, Guoying Zhao

cs.AI

摘要

我們解決了通過控制來自同一人的面部動作單元（AU）的相對變化來進行面部表情編輯的問題。這使我們能夠以精細、連續且可解釋的方式編輯這個特定人的表情，同時保留其身份、姿勢、背景和詳細的面部特徵。我們所提出的模型MagicFace 的關鍵在於一個以AU變化為條件的擴散模型和一個ID編碼器，以保留高一致性的面部細節。具體來說，為了保留具有輸入身份的面部細節，我們利用預訓練的穩定擴散模型的能力，並設計了一個ID編碼器通過自我關注來合併外觀特徵。為了保持背景和姿勢的一致性，我們引入了一個高效的屬性控制器，明確告知模型目標的當前背景和姿勢。通過將AU變化注入去噪UNet，我們的模型可以使用各種AU組合使任意身份動畫化，相對於其他面部表情編輯作品，在高保真度表情編輯方面產生了優越的結果。代碼可在https://github.com/weimengting/MagicFace 公開獲取。

English

We address the problem of facial expression editing by controling the relative variation of facial action-unit (AU) from the same person. This enables us to edit this specific person's expression in a fine-grained, continuous and interpretable manner, while preserving their identity, pose, background and detailed facial attributes. Key to our model, which we dub MagicFace, is a diffusion model conditioned on AU variations and an ID encoder to preserve facial details of high consistency. Specifically, to preserve the facial details with the input identity, we leverage the power of pretrained Stable-Diffusion models and design an ID encoder to merge appearance features through self-attention. To keep background and pose consistency, we introduce an efficient Attribute Controller by explicitly informing the model of current background and pose of the target. By injecting AU variations into a denoising UNet, our model can animate arbitrary identities with various AU combinations, yielding superior results in high-fidelity expression editing compared to other facial expression editing works. Code is publicly available at https://github.com/weimengting/MagicFace.