HUGSIM：一個用於自動駕駛的實時、照片逼真且閉環模擬器

摘要

在過去幾十年裡，自駕車算法在感知、規劃和控制方面取得了顯著進展。然而，評估單個組件並不能完全反映整個系統的性能，凸顯了對更全面的評估方法的需求。這促使了HUGSIM的開發，這是一個用於評估自駕車算法的封閉迴路、逼真且實時的模擬器。我們通過將捕獲的2D RGB圖像通過3D高斯飛濺提升到3D空間，改善了封閉迴路情境的渲染質量，並構建了封閉迴路環境。在渲染方面，我們應對了封閉迴路情境中新視角合成的挑戰，包括視角外推和360度車輛渲染。除了新視角合成，HUGSIM進一步實現了完整的封閉模擬迴路，根據控制命令動態更新自我和參與者的狀態和觀察。此外，HUGSIM提供了來自KITTI-360、Waymo、nuScenes和PandaSet的70多個序列以及400多種不同情境的全面基準，為現有自駕車算法提供了公平且現實的評估平台。HUGSIM不僅作為一個直觀的評估基準，還在逼真的封閉迴路環境中開啟了微調自駕車算法的潛力。

English

In the past few decades, autonomous driving algorithms have made significant progress in perception, planning, and control. However, evaluating individual components does not fully reflect the performance of entire systems, highlighting the need for more holistic assessment methods. This motivates the development of HUGSIM, a closed-loop, photo-realistic, and real-time simulator for evaluating autonomous driving algorithms. We achieve this by lifting captured 2D RGB images into the 3D space via 3D Gaussian Splatting, improving the rendering quality for closed-loop scenarios, and building the closed-loop environment. In terms of rendering, We tackle challenges of novel view synthesis in closed-loop scenarios, including viewpoint extrapolation and 360-degree vehicle rendering. Beyond novel view synthesis, HUGSIM further enables the full closed simulation loop, dynamically updating the ego and actor states and observations based on control commands. Moreover, HUGSIM offers a comprehensive benchmark across more than 70 sequences from KITTI-360, Waymo, nuScenes, and PandaSet, along with over 400 varying scenarios, providing a fair and realistic evaluation platform for existing autonomous driving algorithms. HUGSIM not only serves as an intuitive evaluation benchmark but also unlocks the potential for fine-tuning autonomous driving algorithms in a photorealistic closed-loop setting.

HUGSIM：一個用於自動駕駛的實時、照片逼真且閉環模擬器

HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving

摘要

Summary

Support