HoloPart：生成式三维部件无模态分割

摘要

三维部件全模态分割——将三维形状分解为完整且语义明确的部分，即使在被遮挡的情况下——对于三维内容的创建与理解而言，是一项极具挑战性但至关重要的任务。现有的三维部件分割方法仅能识别可见的表面片段，限制了其应用范围。受二维全模态分割的启发，我们将这一新颖任务引入三维领域，并提出了一种实用的两阶段方法，以应对推断被遮挡三维几何、保持全局形状一致性以及处理有限训练数据下多样化形状的关键挑战。首先，我们利用现有的三维部件分割技术获取初始的不完整部件片段。其次，我们引入了HoloPart，一种基于扩散模型的新颖方法，用于将这些片段补全为完整的三维部件。HoloPart采用了一种特殊架构，结合局部注意力机制以捕捉细粒度的部件几何特征，以及全局形状上下文注意力机制以确保整体形状的一致性。我们基于ABO和PartObjaverse-Tiny数据集引入了新的基准测试，并证明HoloPart显著优于当前最先进的形状补全方法。通过将HoloPart与现有分割技术相结合，我们在三维部件全模态分割上取得了令人鼓舞的成果，为几何编辑、动画制作及材质分配等应用开辟了新的途径。

English

3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.

HoloPart：生成式三维部件无模态分割

HoloPart: Generative 3D Part Amodal Segmentation

摘要

Summary

Support

Support