3DTopia-XL：透過基元擴散實現高質量3D資產生成的規模化

3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion

September 19, 2024

作者: Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, Liang Pan, Dahua Lin, Ziwei Liu

cs.AI

摘要

在各個產業對高品質3D資產日益增加的需求下，迫切需要高效且自動化的3D內容創建。儘管近年來3D生成模型有所進步，現有方法仍面臨優化速度、幾何保真度以及物理渲染資產不足等挑戰。本文介紹了3DTopia-XL，一個可擴展的本地3D生成模型，旨在克服這些限制。3DTopia-XL利用一種新型基於基元的3D表示法PrimX，將詳細形狀、反照率和材質場編碼為緊湊的張量格式，有助於使用PBR資產建模高解析度幾何。除了新的表示法，我們提出了基於擴散Transformer（DiT）的生成框架，包括1）基元補丁壓縮和2）潛在基元擴散。3DTopia-XL學會從文本或視覺輸入生成高品質的3D資產。我們進行了大量定性和定量實驗，證明了3DTopia-XL在生成具有細緻紋理和材料的高品質3D資產方面顯著優於現有方法，有效地彌合了生成模型與實際應用之間的質量差距。

English

The increasing demand for high-quality 3D assets across various industries necessitates efficient and automated 3D content creation. Despite recent advancements in 3D generative models, existing methods still face challenges with optimization speed, geometric fidelity, and the lack of assets for physically based rendering (PBR). In this paper, we introduce 3DTopia-XL, a scalable native 3D generative model designed to overcome these limitations. 3DTopia-XL leverages a novel primitive-based 3D representation, PrimX, which encodes detailed shape, albedo, and material field into a compact tensorial format, facilitating the modeling of high-resolution geometry with PBR assets. On top of the novel representation, we propose a generative framework based on Diffusion Transformer (DiT), which comprises 1) Primitive Patch Compression, 2) and Latent Primitive Diffusion. 3DTopia-XL learns to generate high-quality 3D assets from textual or visual inputs. We conduct extensive qualitative and quantitative experiments to demonstrate that 3DTopia-XL significantly outperforms existing methods in generating high-quality 3D assets with fine-grained textures and materials, efficiently bridging the quality gap between generative models and real-world applications.