Marigold-DC：具有引导扩散的零样本单目深度补全

摘要

深度完成将稀疏深度测量升级为密集深度图，受传统图像引导。针对这一高度不适定任务，现有方法在严格约束的情况下运行，当应用于训练领域之外的图像或可用深度测量稀疏、分布不均匀或密度不同时往往难以应对。受最近单目深度估计进展的启发，我们将深度完成重新构建为由稀疏测量引导的图像条件深度图生成。我们的方法Marigold-DC基于预训练的单目深度估计潜在扩散模型，并通过一个优化方案将深度观测作为测试时引导注入，该方案与去噪扩散的迭代推理同时运行。该方法在各种环境中展现出出色的零样本泛化能力，甚至能有效处理极度稀疏的引导。我们的结果表明，当代单目深度先验极大地加强了深度完成的鲁棒性：更好地将任务视为从（密集）图像像素中恢复密集深度，受稀疏深度引导；而不是将其视为修复（稀疏）深度，受图像引导。项目网站：https://MarigoldDepthCompletion.github.io/

English

Depth completion upgrades sparse depth measurements into dense depth maps guided by a conventional image. Existing methods for this highly ill-posed task operate in tightly constrained settings and tend to struggle when applied to images outside the training domain or when the available depth measurements are sparse, irregularly distributed, or of varying density. Inspired by recent advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation guided by sparse measurements. Our method, Marigold-DC, builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance via an optimization scheme that runs in tandem with the iterative inference of denoising diffusion. The method exhibits excellent zero-shot generalization across a diverse range of environments and handles even extremely sparse guidance effectively. Our results suggest that contemporary monocular depth priors greatly robustify depth completion: it may be better to view the task as recovering dense depth from (dense) image pixels, guided by sparse depth; rather than as inpainting (sparse) depth, guided by an image. Project website: https://MarigoldDepthCompletion.github.io/

Marigold-DC：具有引导扩散的零样本单目深度补全

Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion

摘要

Summary

Support