序列重要性：利用視頻模型進行3D超分辨率

摘要

3D 超分辨率的目標是從低解析度（LR）多視角影像中重建高保真度的 3D 模型。早期研究主要集中在單圖像超分辨率（SISR）模型上，將 LR 圖像升頻至高解析度圖像。然而，這些方法通常缺乏視角一致性，因為它們獨立地對每個圖像進行操作。儘管已廣泛探索各種後處理技術來減輕這些不一致性，但它們尚未完全解決問題。在本文中，我們通過利用視頻超分辨率（VSR）模型，對 3D 超分辨率進行了全面研究。通過利用 VSR 模型，我們確保更高程度的空間一致性，並可以參考周圍的空間信息，從而實現更準確和詳細的重建。我們的研究結果顯示，即使在缺乏精確空間對齊的序列上，VSR 模型也能表現出色。基於這一觀察，我們提出了一種簡單而實用的方法，用於對齊 LR 圖像，而無需進行微調或從訓練的 3D 模型上生成“平滑”軌跡。實驗結果表明，這些驚人簡單的算法可以在標準基準數據集（如 NeRF-synthetic 和 MipNeRF-360 數據集）上實現 3D 超分辨率任務的最新成果。項目頁面：https://ko-lani.github.io/Sequence-Matters

English

3D super-resolution aims to reconstruct high-fidelity 3D models from low-resolution (LR) multi-view images. Early studies primarily focused on single-image super-resolution (SISR) models to upsample LR images into high-resolution images. However, these methods often lack view consistency because they operate independently on each image. Although various post-processing techniques have been extensively explored to mitigate these inconsistencies, they have yet to fully resolve the issues. In this paper, we perform a comprehensive study of 3D super-resolution by leveraging video super-resolution (VSR) models. By utilizing VSR models, we ensure a higher degree of spatial consistency and can reference surrounding spatial information, leading to more accurate and detailed reconstructions. Our findings reveal that VSR models can perform remarkably well even on sequences that lack precise spatial alignment. Given this observation, we propose a simple yet practical approach to align LR images without involving fine-tuning or generating 'smooth' trajectory from the trained 3D models over LR images. The experimental results show that the surprisingly simple algorithms can achieve the state-of-the-art results of 3D super-resolution tasks on standard benchmark datasets, such as the NeRF-synthetic and MipNeRF-360 datasets. Project page: https://ko-lani.github.io/Sequence-Matters

序列重要性：利用視頻模型進行3D超分辨率

Sequence Matters: Harnessing Video Models in 3D Super-Resolution

摘要

Summary

Support