PE3R：感知高效的三维重建

PE3R: Perception-Efficient 3D Reconstruction

March 10, 2025

作者: Jie Hu, Shizun Wang, Xinchao Wang

cs.AI

摘要

近期，二维到三维感知技术的进步显著提升了对二维图像中三维场景的理解能力。然而，现有方法面临诸多关键挑战，包括跨场景泛化能力有限、感知精度欠佳以及重建速度缓慢。为克服这些局限，我们提出了感知高效三维重建框架（PE3R），旨在同时提升准确性与效率。PE3R采用前馈架构，实现了快速的三维语义场重建。该框架在多样化的场景与对象上展现出强大的零样本泛化能力，并显著提高了重建速度。在二维到三维开放词汇分割及三维重建上的大量实验验证了PE3R的有效性与多功能性。该框架在三维语义场重建中实现了至少9倍的加速，同时在感知精度与重建精确度上取得显著提升，为领域树立了新标杆。代码已公开于：https://github.com/hujiecpp/PE3R。

English

Recent advancements in 2D-to-3D perception have significantly improved the understanding of 3D scenes from 2D images. However, existing methods face critical challenges, including limited generalization across scenes, suboptimal perception accuracy, and slow reconstruction speeds. To address these limitations, we propose Perception-Efficient 3D Reconstruction (PE3R), a novel framework designed to enhance both accuracy and efficiency. PE3R employs a feed-forward architecture to enable rapid 3D semantic field reconstruction. The framework demonstrates robust zero-shot generalization across diverse scenes and objects while significantly improving reconstruction speed. Extensive experiments on 2D-to-3D open-vocabulary segmentation and 3D reconstruction validate the effectiveness and versatility of PE3R. The framework achieves a minimum 9-fold speedup in 3D semantic field reconstruction, along with substantial gains in perception accuracy and reconstruction precision, setting new benchmarks in the field. The code is publicly available at: https://github.com/hujiecpp/PE3R.