ChatGen：从自由对话生成自动文本到图像

ChatGen: Automatic Text-to-Image Generation From FreeStyle Chatting

November 26, 2024

作者: Chengyou Jia, Changliang Xia, Zhuohang Dang, Weijia Wu, Hangwei Qian, Minnan Luo

cs.AI

摘要

尽管文本到图像（T2I）生成模型取得了显著进展，用户在实际场景中经常面临试错挑战。这一挑战源自于繁琐步骤的复杂性和不确定性，如制作合适的提示、选择适当的模型和配置特定参数，使用户不得不进行劳动密集型尝试以获得期望的图像。本文提出了自动T2I生成，旨在自动化这些繁琐步骤，使用户可以用自由聊天的方式简单描述他们的需求。为了系统研究这一问题，我们首先介绍了ChatGenBench，这是一个专为自动T2I设计的新型基准。它具有高质量的配对数据和多样的自由输入，可以全面评估自动T2I模型在所有步骤上的表现。此外，我们将自动T2I视为一个复杂的多步推理任务，提出了ChatGen-Evo，这是一种多阶段演化策略，逐步赋予模型必要的自动化技能。通过对步骤准确性和图像质量的广泛评估，ChatGen-Evo显著提升了各种基准的性能。我们的评估还揭示了推进自动T2I的宝贵见解。我们所有的数据、代码和模型将在https://chengyou-jia.github.io/ChatGen-Home 上提供。

English

Despite the significant advancements in text-to-image (T2I) generative models, users often face a trial-and-error challenge in practical scenarios. This challenge arises from the complexity and uncertainty of tedious steps such as crafting suitable prompts, selecting appropriate models, and configuring specific arguments, making users resort to labor-intensive attempts for desired images. This paper proposes Automatic T2I generation, which aims to automate these tedious steps, allowing users to simply describe their needs in a freestyle chatting way. To systematically study this problem, we first introduce ChatGenBench, a novel benchmark designed for Automatic T2I. It features high-quality paired data with diverse freestyle inputs, enabling comprehensive evaluation of automatic T2I models across all steps. Additionally, recognizing Automatic T2I as a complex multi-step reasoning task, we propose ChatGen-Evo, a multi-stage evolution strategy that progressively equips models with essential automation skills. Through extensive evaluation across step-wise accuracy and image quality, ChatGen-Evo significantly enhances performance over various baselines. Our evaluation also uncovers valuable insights for advancing automatic T2I. All our data, code, and models will be available in https://chengyou-jia.github.io/ChatGen-Home