Chirpy3D：用於創意3D鳥類生成的連續部分潛在特徵

Chirpy3D: Continuous Part Latents for Creative 3D Bird Generation

January 7, 2025

作者: Kam Woh Ng, Jing Yang, Jia Wei Sii, Jiankang Deng, Chee Seng Chan, Yi-Zhe Song, Tao Xiang, Xiatian Zhu

cs.AI

摘要

在本文中，我們將精細的3D生成的範疇推進至真正具有創意的領域。目前的方法要麼缺乏細緻的細節，要麼僅僅模仿現有的物體 - 我們實現了兩者兼具。通過將2D細緻理解提升至3D，通過多視圖擴散和將部分潛在因素建模為連續分佈，我們解鎖了通過插值和抽樣生成全新但合理部分的能力。自監督特徵一致性損失進一步確保了這些未曾見過部分的穩定生成。結果是第一個能夠創建具有超越現有示例的物種特定細節的新穎3D物體的系統。雖然我們在鳥類上展示了我們的方法，但基礎框架超越了能夠鳴叫的事物！代碼將在https://github.com/kamwoh/chirpy3d 上發布。

English

In this paper, we push the boundaries of fine-grained 3D generation into truly creative territory. Current methods either lack intricate details or simply mimic existing objects -- we enable both. By lifting 2D fine-grained understanding into 3D through multi-view diffusion and modeling part latents as continuous distributions, we unlock the ability to generate entirely new, yet plausible parts through interpolation and sampling. A self-supervised feature consistency loss further ensures stable generation of these unseen parts. The result is the first system capable of creating novel 3D objects with species-specific details that transcend existing examples. While we demonstrate our approach on birds, the underlying framework extends beyond things that can chirp! Code will be released at https://github.com/kamwoh/chirpy3d.