DINO-WM：基于预训练视觉特征的世界模型实现零样本规划

DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning

November 7, 2024

作者: Gaoyue Zhou, Hengkai Pan, Yann LeCun, Lerrel Pinto

cs.AI

摘要

预测未来结果的能力在物理推理中至关重要。然而，这种被称为世界模型的预测模型往往难以学习，并且通常是为具有在线策略学习的特定任务解决方案而开发的。我们认为，世界模型的真正潜力在于其能够利用仅有的被动数据进行跨多样问题的推理和规划。具体而言，我们要求世界模型具备以下三个特性：1）能够在离线、预先收集的轨迹上进行训练，2）支持测试时行为优化，3）促进任务无关的推理。为了实现这一目标，我们提出了DINO世界模型（DINO-WM），这是一种新的方法，可以在不重建视觉世界的情况下对视觉动态进行建模。DINO-WM利用使用DINOv2预训练的空间块特征，使其能够通过预测未来块特征来从离线行为轨迹中学习。这种设计使得DINO-WM能够通过将期望的目标块特征视为预测目标，通过行动序列优化实现观察目标。我们在各个领域对DINO-WM进行了评估，包括迷宫导航、桌面推动和粒子操纵。我们的实验表明，DINO-WM能够在测试时生成零样本行为解决方案，而无需依赖专家演示、奖励建模或预先学习的逆模型。值得注意的是，与先前的最新工作相比，DINO-WM表现出强大的泛化能力，适应各种任务系列，如任意配置的迷宫、带有不同物体形状的推动操纵以及多粒子场景。

English

The ability to predict future outcomes given control actions is fundamental for physical reasoning. However, such predictive models, often called world models, have proven challenging to learn and are typically developed for task-specific solutions with online policy learning. We argue that the true potential of world models lies in their ability to reason and plan across diverse problems using only passive data. Concretely, we require world models to have the following three properties: 1) be trainable on offline, pre-collected trajectories, 2) support test-time behavior optimization, and 3) facilitate task-agnostic reasoning. To realize this, we present DINO World Model (DINO-WM), a new method to model visual dynamics without reconstructing the visual world. DINO-WM leverages spatial patch features pre-trained with DINOv2, enabling it to learn from offline behavioral trajectories by predicting future patch features. This design allows DINO-WM to achieve observational goals through action sequence optimization, facilitating task-agnostic behavior planning by treating desired goal patch features as prediction targets. We evaluate DINO-WM across various domains, including maze navigation, tabletop pushing, and particle manipulation. Our experiments demonstrate that DINO-WM can generate zero-shot behavioral solutions at test time without relying on expert demonstrations, reward modeling, or pre-learned inverse models. Notably, DINO-WM exhibits strong generalization capabilities compared to prior state-of-the-art work, adapting to diverse task families such as arbitrarily configured mazes, push manipulation with varied object shapes, and multi-particle scenarios.