Disco4D：從單張圖像生成和動畫化解耦的4D人類模型

Disco4D: Disentangled 4D Human Generation and Animation from a Single Image

September 25, 2024

作者: Hui En Pang, Shuai Liu, Zhongang Cai, Lei Yang, Tianwei Zhang, Ziwei Liu

cs.AI

摘要

我們提出了Disco4D，一種新穎的高斯點陣框架，用於從單張圖像生成和動畫化4D人物。與現有方法不同，Disco4D明顯地將服裝（使用高斯模型）與人體（使用SMPL-X模型）分離，顯著增強了生成細節和靈活性。它具有以下技術創新。 1) Disco4D 學習有效地將服裝高斯模型套用到 SMPL-X 高斯模型上。2) 它採用擴散模型來增強3D生成過程，例如，對於輸入圖像中不可見的遮蔽部分進行建模。3) 它為每個服裝高斯模型學習身份編碼，以促進服裝資產的分離和提取。 此外，Disco4D 自然地支持具有生動動態的4D人物動畫。大量實驗證明了Disco4D在4D人物生成和動畫任務上的優越性。我們的可視化結果可以在 https://disco-4d.github.io/ 找到。

English

We present Disco4D, a novel Gaussian Splatting framework for 4D human generation and animation from a single image. Different from existing methods, Disco4D distinctively disentangles clothings (with Gaussian models) from the human body (with SMPL-X model), significantly enhancing the generation details and flexibility. It has the following technical innovations. 1) Disco4D learns to efficiently fit the clothing Gaussians over the SMPL-X Gaussians. 2) It adopts diffusion models to enhance the 3D generation process, e.g., modeling occluded parts not visible in the input image. 3) It learns an identity encoding for each clothing Gaussian to facilitate the separation and extraction of clothing assets. Furthermore, Disco4D naturally supports 4D human animation with vivid dynamics. Extensive experiments demonstrate the superiority of Disco4D on 4D human generation and animation tasks. Our visualizations can be found in https://disco-4d.github.io/.