DreamPolish：逐步幾何生成的領域分數蒸餾

DreamPolish: Domain Score Distillation With Progressive Geometry Generation

November 3, 2024

作者: Yean Cheng, Ziqi Cai, Ming Ding, Wendi Zheng, Shiyu Huang, Yuxiao Dong, Jie Tang, Boxin Shi

cs.AI

摘要

我們介紹了DreamPolish，一個在生成精緻幾何和高質量紋理方面表現出色的文本到3D生成模型。在幾何構建階段，我們的方法利用多個神經表示來增強合成過程的穩定性。我們不僅依賴於新樣本視圖中的視圖條件擴散先驗，這通常會導致幾何表面上的不良藝術品，而是在基於視點的不同視野的情況下，加入額外的法向量估計器來修飾幾何細節。我們建議添加一個表面拋光階段，只需進行少量訓練步驟，就能有效地改進由於前幾個階段的有限引導而產生的藝術品，並產生具有更理想幾何的3D物體。在使用預訓練文本到圖像模型進行紋理生成的關鍵主題是在這些模型的廣泛潛在分佈中找到一個包含照片逼真和一致渲染的合適領域。在紋理生成階段，我們引入了一個新穎的分數蒸餾目標，即領域分數蒸餾（DSD），來引導神經表示朝向這樣的領域。我們從文本條件圖像生成任務中的無分類器引導（CFG）中汲取靈感，並展示CFG和變分分佈引導在梯度引導中代表不同方面，對於提高紋理質量都是至關重要的領域。大量實驗表明，我們提出的模型可以生成具有拋光表面和照片逼真紋理的3D資產，勝過現有的最先進方法。

English

We introduce DreamPolish, a text-to-3D generation model that excels in producing refined geometry and high-quality textures. In the geometry construction phase, our approach leverages multiple neural representations to enhance the stability of the synthesis process. Instead of relying solely on a view-conditioned diffusion prior in the novel sampled views, which often leads to undesired artifacts in the geometric surface, we incorporate an additional normal estimator to polish the geometry details, conditioned on viewpoints with varying field-of-views. We propose to add a surface polishing stage with only a few training steps, which can effectively refine the artifacts attributed to limited guidance from previous stages and produce 3D objects with more desirable geometry. The key topic of texture generation using pretrained text-to-image models is to find a suitable domain in the vast latent distribution of these models that contains photorealistic and consistent renderings. In the texture generation phase, we introduce a novel score distillation objective, namely domain score distillation (DSD), to guide neural representations toward such a domain. We draw inspiration from the classifier-free guidance (CFG) in textconditioned image generation tasks and show that CFG and variational distribution guidance represent distinct aspects in gradient guidance and are both imperative domains for the enhancement of texture quality. Extensive experiments show our proposed model can produce 3D assets with polished surfaces and photorealistic textures, outperforming existing state-of-the-art methods.