Marigold-DC：具有引導擴散的零樣本單目深度完成。

摘要

深度完成將稀疏深度測量升級為由傳統影像引導的密集深度地圖。針對這個高度不透明的任務，現有方法在嚴格受限的環境中運作，當應用於訓練領域之外的影像，或當可用的深度測量是稀疏、不規則分佈或密度不一致時，往往會遇到困難。受最近單眼深度估計進展的啟發，我們將深度完成重新定義為由稀疏測量引導的影像條件深度地圖生成。我們的方法Marigold-DC基於預訓練的潛在擴散模型進行單眼深度估計，並通過優化方案將深度觀測作為測試時的引導，與去噪擴散的迭代推斷同步運行。該方法在各種環境中展現出優異的零樣本泛化能力，並有效處理極度稀疏的引導。我們的結果表明，當代單眼深度先驗極大地增強了深度完成的穩健性：將任務視為從（密集）影像像素中恢復密集深度，由稀疏深度引導；而不是將其視為由影像引導的修補（稀疏）深度。項目網站：https://MarigoldDepthCompletion.github.io/

English

Depth completion upgrades sparse depth measurements into dense depth maps guided by a conventional image. Existing methods for this highly ill-posed task operate in tightly constrained settings and tend to struggle when applied to images outside the training domain or when the available depth measurements are sparse, irregularly distributed, or of varying density. Inspired by recent advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation guided by sparse measurements. Our method, Marigold-DC, builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance via an optimization scheme that runs in tandem with the iterative inference of denoising diffusion. The method exhibits excellent zero-shot generalization across a diverse range of environments and handles even extremely sparse guidance effectively. Our results suggest that contemporary monocular depth priors greatly robustify depth completion: it may be better to view the task as recovering dense depth from (dense) image pixels, guided by sparse depth; rather than as inpainting (sparse) depth, guided by an image. Project website: https://MarigoldDepthCompletion.github.io/

Marigold-DC：具有引導擴散的零樣本單目深度完成。

Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion

摘要

Summary

Support