PanoDreamer：从单张图像合成3D全景图

PanoDreamer: 3D Panorama Synthesis from a Single Image

December 6, 2024

作者: Avinash Paliwal, Xilong Zhou, Andrii Tsarov, Nima Khademi Kalantari

cs.AI

摘要

在本文中，我们提出了PanoDreamer，一种从单个输入图像生成连贯的360度三维场景的新方法。与现有的按顺序生成场景的方法不同，我们将问题构建为单图全景和深度估计。一旦获得了连贯的全景图像及其对应的深度，就可以通过修补小遮挡区域并将其投影到三维空间中来重建场景。我们的关键贡献在于将单图全景和深度估计形式化为两个优化任务，并引入交替最小化策略来有效解决它们的目标。我们证明了我们的方法在单图360度场景重建方面在一致性和整体质量方面优于现有技术。

English

In this paper, we present PanoDreamer, a novel method for producing a coherent 360^circ 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360^circ scene reconstruction in terms of consistency and overall quality.