Chirpy3D：用于创意3D鸟类生成的连续部分潜变量

Chirpy3D: Continuous Part Latents for Creative 3D Bird Generation

January 7, 2025

作者: Kam Woh Ng, Jing Yang, Jia Wei Sii, Jiankang Deng, Chee Seng Chan, Yi-Zhe Song, Tao Xiang, Xiatian Zhu

cs.AI

摘要

本文将精细化的三维生成推向真正创意的领域边界。当前方法要么缺乏复杂细节，要么仅仅模仿现有对象——我们实现了两者兼具。通过将二维精细理解提升至三维，通过多视角扩散和对部分潜在因素建模为连续分布，我们解锁了通过插值和抽样生成全新但可信的部分的能力。自监督特征一致性损失进一步确保了这些未曾见过部分的稳定生成。其结果是第一个能够创造具有超越现有示例的物种特定细节的全新三维对象的系统。虽然我们在鸟类上展示了我们的方法，但基础框架超越了那些能够鸣叫的事物！代码将在 https://github.com/kamwoh/chirpy3d 上发布。

English

In this paper, we push the boundaries of fine-grained 3D generation into truly creative territory. Current methods either lack intricate details or simply mimic existing objects -- we enable both. By lifting 2D fine-grained understanding into 3D through multi-view diffusion and modeling part latents as continuous distributions, we unlock the ability to generate entirely new, yet plausible parts through interpolation and sampling. A self-supervised feature consistency loss further ensures stable generation of these unseen parts. The result is the first system capable of creating novel 3D objects with species-specific details that transcend existing examples. While we demonstrate our approach on birds, the underlying framework extends beyond things that can chirp! Code will be released at https://github.com/kamwoh/chirpy3d.