场景中心的无监督全景分割

Scene-Centric Unsupervised Panoptic Segmentation

April 2, 2025

作者: Oliver Hahn, Christoph Reich, Nikita Araslanov, Daniel Cremers, Christian Rupprecht, Stefan Roth

cs.AI

摘要

无监督全景分割旨在无需依赖人工标注数据进行训练的情况下，将图像划分为具有语义意义的区域和独立的对象实例。与以往的无监督全景场景理解研究不同，我们摒弃了对以对象为中心的训练数据的依赖，从而实现了对复杂场景的无监督理解。为此，我们提出了首个直接在场景中心图像上进行训练的无监督全景分割方法。具体而言，我们提出了一种结合视觉表征、深度和运动线索的方法，以在复杂的场景中心数据上获取高分辨率全景伪标签。通过伪标签训练与全景自训练策略的结合，我们开发了一种新颖的方法，能够准确预测复杂场景的全景分割，而无需任何人工标注。我们的方法显著提升了全景分割的质量，例如，在Cityscapes数据集上的无监督全景分割任务中，以9.4%的PQ分数超越了当前的最新技术水平。

English

Unsupervised panoptic segmentation aims to partition an image into semantically meaningful regions and distinct object instances without training on manually annotated data. In contrast to prior work on unsupervised panoptic scene understanding, we eliminate the need for object-centric training data, enabling the unsupervised understanding of complex scenes. To that end, we present the first unsupervised panoptic method that directly trains on scene-centric imagery. In particular, we propose an approach to obtain high-resolution panoptic pseudo labels on complex scene-centric data, combining visual representations, depth, and motion cues. Utilizing both pseudo-label training and a panoptic self-training strategy yields a novel approach that accurately predicts panoptic segmentation of complex scenes without requiring any human annotations. Our approach significantly improves panoptic quality, e.g., surpassing the recent state of the art in unsupervised panoptic segmentation on Cityscapes by 9.4% points in PQ.