序列至关重要：利用视频模型进行3D超分辨率。

摘要

3D超分辨率旨在从低分辨率（LR）多视图图像中重建高保真度的3D模型。早期研究主要集中在单图像超分辨率（SISR）模型上，将LR图像上采样为高分辨率图像。然而，这些方法通常缺乏视角一致性，因为它们独立地处理每个图像。尽管已广泛探讨了各种后处理技术来减轻这些不一致性，但它们尚未完全解决问题。在本文中，我们通过利用视频超分辨率（VSR）模型，对3D超分辨率进行了全面研究。通过利用VSR模型，我们确保了更高程度的空间一致性，并可以参考周围的空间信息，从而实现更准确和详细的重建。我们的研究结果表明，即使在缺乏精确空间对齐的序列上，VSR模型也能表现出色。基于这一观察，我们提出了一种简单而实用的方法，用于对齐LR图像，而无需进行微调或从训练好的3D模型在LR图像上生成“平滑”轨迹。实验结果表明，这些令人惊讶的简单算法能够在标准基准数据集（如NeRF-synthetic和MipNeRF-360数据集）上实现3D超分辨率任务的最新成果。项目页面：https://ko-lani.github.io/Sequence-Matters

English

3D super-resolution aims to reconstruct high-fidelity 3D models from low-resolution (LR) multi-view images. Early studies primarily focused on single-image super-resolution (SISR) models to upsample LR images into high-resolution images. However, these methods often lack view consistency because they operate independently on each image. Although various post-processing techniques have been extensively explored to mitigate these inconsistencies, they have yet to fully resolve the issues. In this paper, we perform a comprehensive study of 3D super-resolution by leveraging video super-resolution (VSR) models. By utilizing VSR models, we ensure a higher degree of spatial consistency and can reference surrounding spatial information, leading to more accurate and detailed reconstructions. Our findings reveal that VSR models can perform remarkably well even on sequences that lack precise spatial alignment. Given this observation, we propose a simple yet practical approach to align LR images without involving fine-tuning or generating 'smooth' trajectory from the trained 3D models over LR images. The experimental results show that the surprisingly simple algorithms can achieve the state-of-the-art results of 3D super-resolution tasks on standard benchmark datasets, such as the NeRF-synthetic and MipNeRF-360 datasets. Project page: https://ko-lani.github.io/Sequence-Matters

序列至关重要：利用视频模型进行3D超分辨率。

Sequence Matters: Harnessing Video Models in 3D Super-Resolution

摘要

Summary

Support