DreamDPO：通过直接偏好优化实现文本到3D生成与人类偏好的对齐

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization

February 5, 2025

作者: Zhenglin Zhou, Xiaobo Xia, Fan Ma, Hehe Fan, Yi Yang, Tat-Seng Chua

cs.AI

摘要

文本到3D生成自动化地从文本描述中创建3D内容，这在各个领域具有变革性潜力。然而，现有方法常常难以与人类偏好对齐，从而限制了它们的适用性和灵活性。为了解决这些限制，在本文中，我们提出了DreamDPO，这是一个基于优化的框架，将人类偏好整合到3D生成过程中，通过直接偏好优化。在实践中，DreamDPO首先构建成对示例，然后使用奖励或大型多模态模型比较它们与人类偏好的对齐情况，最后通过偏好驱动的损失函数优化3D表示。通过利用成对比较来反映偏好，DreamDPO减少了对精确点对点质量评估的依赖，同时通过偏好引导的优化实现了细粒度可控性。实验证明，DreamDPO取得了竞争性的结果，与现有方法相比提供了更高质量和更可控的3D内容。代码和模型将开源。

English

Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle to align generated content with human preferences, limiting their applicability and flexibility. To address these limitations, in this paper, we propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process, through direct preference optimization. Practically, DreamDPO first constructs pairwise examples, then compare their alignment with human preferences using reward or large multimodal models, and lastly optimizes the 3D representation with a preference-driven loss function. By leveraging pairwise comparison to reflect preferences, DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability through preference-guided optimization. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content compared to existing methods. The code and models will be open-sourced.