PanoDreamer：從單張影像合成3D全景图

PanoDreamer: 3D Panorama Synthesis from a Single Image

December 6, 2024

作者: Avinash Paliwal, Xilong Zhou, Andrii Tsarov, Nima Khademi Kalantari

cs.AI

摘要

本文介紹了一種名為PanoDreamer的新方法，用於從單張輸入圖像生成一個連貫的360度3D場景。與現有的按順序生成場景的方法不同，我們將問題定義為單圖全景和深度估計。一旦獲得了連貫的全景圖像及其對應的深度，就可以通過修補小區域的遮擋部分並將其投影到3D空間中來重建場景。我們的主要貢獻在於將單圖全景和深度估計定義為兩個優化任務，並引入交替最小化策略來有效解決它們的目標。我們證明了我們的方法在單圖360度場景重建方面在一致性和整體質量方面優於現有技術。

English

In this paper, we present PanoDreamer, a novel method for producing a coherent 360^circ 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360^circ scene reconstruction in terms of consistency and overall quality.