HoloPart：生成式三維部件無遮擋分割

HoloPart: Generative 3D Part Amodal Segmentation

April 10, 2025

作者: Yunhan Yang, Yuan-Chen Guo, Yukun Huang, Zi-Xin Zou, Zhipeng Yu, Yangguang Li, Yan-Pei Cao, Xihui Liu

cs.AI

摘要

三維部件全模分割——將三維形體分解為完整且語義明確的部件，即便在遮擋情況下——對於三維內容的創建與理解而言，是一項既具挑戰性又至關重要的任務。現有的三維部件分割方法僅能識別可見的表面片段，限制了其實用性。受二維全模分割的啟發，我們將這一新穎任務引入三維領域，並提出了一種實用的兩階段方法，以應對推斷被遮擋三維幾何、保持全局形狀一致性以及在有限訓練數據下處理多樣化形狀等關鍵挑戰。首先，我們利用現有的三維部件分割技術獲取初始的不完整部件片段。其次，我們引入了HoloPart，一種基於擴散模型的新穎方法，用於將這些片段補全為完整的三維部件。HoloPart採用了一種特殊架構，結合局部注意力機制以捕捉細粒度的部件幾何，以及全局形狀上下文注意力機制以確保整體形狀的一致性。我們基於ABO和PartObjaverse-Tiny數據集引入了新的基準測試，並證明HoloPart顯著優於最先進的形狀補全方法。通過將HoloPart與現有的分割技術相結合，我們在三維部件全模分割上取得了令人鼓舞的成果，為幾何編輯、動畫製作及材質分配等應用開辟了新的途徑。

English

3D part amodal segmentation--decomposing a 3D shape into complete, semantically meaningful parts, even when occluded--is a challenging but crucial task for 3D content creation and understanding. Existing 3D part segmentation methods only identify visible surface patches, limiting their utility. Inspired by 2D amodal segmentation, we introduce this novel task to the 3D domain and propose a practical, two-stage approach, addressing the key challenges of inferring occluded 3D geometry, maintaining global shape consistency, and handling diverse shapes with limited training data. First, we leverage existing 3D part segmentation to obtain initial, incomplete part segments. Second, we introduce HoloPart, a novel diffusion-based model, to complete these segments into full 3D parts. HoloPart utilizes a specialized architecture with local attention to capture fine-grained part geometry and global shape context attention to ensure overall shape consistency. We introduce new benchmarks based on the ABO and PartObjaverse-Tiny datasets and demonstrate that HoloPart significantly outperforms state-of-the-art shape completion methods. By incorporating HoloPart with existing segmentation techniques, we achieve promising results on 3D part amodal segmentation, opening new avenues for applications in geometry editing, animation, and material assignment.