个性化三维生成化身：从单一肖像生成

PERSE: Personalized 3D Generative Avatars from A Single Portrait

December 30, 2024

作者: Hyunsoo Cha, Inhee Lee, Hanbyul Joo

cs.AI

摘要

我们提出了PERSE，这是一种从参考肖像构建可动画个性化生成化身的方法。我们的化身模型能够在连续且解缠的潜在空间中进行面部属性编辑，以控制每个面部属性，同时保留个体的身份特征。为实现这一目标，我们的方法首先通过合成大规模的2D视频数据集，其中每个视频包含面部表情和视角的一致变化，结合原始输入中特定面部属性的变化。我们提出了一种新颖的流程，用于生成具有面部属性编辑的高质量、照片级2D视频。利用这一合成属性数据集，我们提出了一种基于3D高斯散射的个性化化身创建方法，学习连续且解缠的潜在空间，以进行直观的面部属性操作。为了在这一潜在空间中实现平滑过渡，我们引入了一种潜在空间正则化技术，通过使用插值的2D面部作为监督。与先前的方法相比，我们展示了PERSE生成具有插值属性的高质量化身，同时保留了参考人物的身份特征。

English

We present PERSE, a method for building an animatable personalized generative avatar from a reference portrait. Our avatar model enables facial attribute editing in a continuous and disentangled latent space to control each facial attribute, while preserving the individual's identity. To achieve this, our method begins by synthesizing large-scale synthetic 2D video datasets, where each video contains consistent changes in the facial expression and viewpoint, combined with a variation in a specific facial attribute from the original input. We propose a novel pipeline to produce high-quality, photorealistic 2D videos with facial attribute editing. Leveraging this synthetic attribute dataset, we present a personalized avatar creation method based on the 3D Gaussian Splatting, learning a continuous and disentangled latent space for intuitive facial attribute manipulation. To enforce smooth transitions in this latent space, we introduce a latent space regularization technique by using interpolated 2D faces as supervision. Compared to previous approaches, we demonstrate that PERSE generates high-quality avatars with interpolated attributes while preserving identity of reference person.