GarVerseLOD：使用具有多个细节级别的数据集从单个野外图像进行高保真度的3D服装重建

GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details

November 5, 2024

作者: Zhongjin Luo, Haolin Liu, Chenghong Li, Wanghao Du, Zirong Jin, Wanhu Sun, Yinyu Nie, Weikai Chen, Xiaoguang Han

cs.AI

摘要

神经隐式函数为从多个甚至单个图像实现服装人体数字化的最新技术带来了令人瞩目的进展。然而，尽管取得了进展，当前的技术仍然难以推广到具有复杂布料变形和身体姿势的未见图像。在这项工作中，我们提出了GarVerseLOD，这是一个新的数据集和框架，为从单个无约束图像实现高保真度3D服装重建铺平了道路。受大型生成模型最近取得的成功启发，我们认为解决泛化挑战的关键之一在于3D服装数据的数量和质量。为此，GarVerseLOD收集了由专业艺术家手动创建的具有细粒度几何细节的6,000个高质量布料模型。除了训练数据的规模外，我们观察到几何解耦的粒度可以在提升学习模型的泛化能力和推理准确性方面发挥重要作用。因此，我们将GarVerseLOD设计为一个具有不同细节级别（LOD）的分层数据集，从无细节风格化形状到融合姿势的服装并具有像素对齐细节。这使我们可以通过将推理分解为更容易的任务，每个任务缩小搜索空间，从而使这个高度不受约束的问题变得可解。为了确保GarVerseLOD能够很好地推广到野外图像，我们提出了一种基于条件扩散模型的新型标记范式，为每个服装模型生成大量具有高照相逼真度的配对图像。我们在大量野外图像上评估了我们的方法。实验结果表明，GarVerseLOD能够生成具有明显更好质量的独立服装部件，优于先前的方法。项目页面：https://garverselod.github.io/

English

Neural implicit functions have brought impressive advances to the state-of-the-art of clothed human digitization from multiple or even single images. However, despite the progress, current arts still have difficulty generalizing to unseen images with complex cloth deformation and body poses. In this work, we present GarVerseLOD, a new dataset and framework that paves the way to achieving unprecedented robustness in high-fidelity 3D garment reconstruction from a single unconstrained image. Inspired by the recent success of large generative models, we believe that one key to addressing the generalization challenge lies in the quantity and quality of 3D garment data. Towards this end, GarVerseLOD collects 6,000 high-quality cloth models with fine-grained geometry details manually created by professional artists. In addition to the scale of training data, we observe that having disentangled granularities of geometry can play an important role in boosting the generalization capability and inference accuracy of the learned model. We hence craft GarVerseLOD as a hierarchical dataset with levels of details (LOD), spanning from detail-free stylized shape to pose-blended garment with pixel-aligned details. This allows us to make this highly under-constrained problem tractable by factorizing the inference into easier tasks, each narrowed down with smaller searching space. To ensure GarVerseLOD can generalize well to in-the-wild images, we propose a novel labeling paradigm based on conditional diffusion models to generate extensive paired images for each garment model with high photorealism. We evaluate our method on a massive amount of in-the-wild images. Experimental results demonstrate that GarVerseLOD can generate standalone garment pieces with significantly better quality than prior approaches. Project page: https://garverselod.github.io/