MaRI：跨领域材料检索集成

MaRI: Material Retrieval Integration across Domains

March 11, 2025

作者: Jianhui Wang, Zhifei Yang, Yangfan He, Huixiong Zhang, Yuxuan Chen, Jingwei Huang

cs.AI

摘要

精确的材料检索对于创建逼真的3D资产至关重要。现有方法依赖于捕捉形状不变和光照变化的材料表示的数据集，这些数据集稀缺且面临多样性有限和现实世界泛化能力不足的挑战。当前大多数方法采用传统的图像搜索技术，它们在捕捉材料空间独特属性方面表现欠佳，导致检索任务性能不佳。针对这些挑战，我们提出了MaRI框架，旨在弥合合成材料与真实世界材料之间的特征空间差距。MaRI通过联合训练图像编码器和材料编码器，采用对比学习策略构建了一个共享嵌入空间，该空间协调了视觉和材料属性，使相似的材料和图像在特征空间中更接近，同时分离不相似的对。为此，我们构建了一个全面的数据集，包含高质量合成材料，这些材料在受控的形状变化和多样光照条件下渲染，以及使用材料传递技术处理和标准化的真实世界材料。大量实验表明，MaRI在多样且复杂的材料检索任务中表现出卓越的性能、准确性和泛化能力，超越了现有方法。

English

Accurate material retrieval is critical for creating realistic 3D assets. Existing methods rely on datasets that capture shape-invariant and lighting-varied representations of materials, which are scarce and face challenges due to limited diversity and inadequate real-world generalization. Most current approaches adopt traditional image search techniques. They fall short in capturing the unique properties of material spaces, leading to suboptimal performance in retrieval tasks. Addressing these challenges, we introduce MaRI, a framework designed to bridge the feature space gap between synthetic and real-world materials. MaRI constructs a shared embedding space that harmonizes visual and material attributes through a contrastive learning strategy by jointly training an image and a material encoder, bringing similar materials and images closer while separating dissimilar pairs within the feature space. To support this, we construct a comprehensive dataset comprising high-quality synthetic materials rendered with controlled shape variations and diverse lighting conditions, along with real-world materials processed and standardized using material transfer techniques. Extensive experiments demonstrate the superior performance, accuracy, and generalization capabilities of MaRI across diverse and complex material retrieval tasks, outperforming existing methods.