多模态LLM可以进行零样本美学推理。

Multimodal LLMs Can Reason about Aesthetics in Zero-Shot

January 15, 2025

作者: Ruixiang Jiang, Changwen Chen

cs.AI

摘要

我们提出了第一项关于如何引发多模态语言模型（MLLMs）推理能力以评估艺术作品美学的研究。为了促进这一调查，我们构建了MM-StyleBench，这是一个用于艺术风格化基准测试的新颖高质量数据集。然后，我们开发了一种基于原则的人类偏好建模方法，并对MLLMs的响应与人类偏好之间的系统相关性进行了分析。我们的实验揭示了MLLMs在艺术评估中存在的固有幻觉问题，与响应主观性有关。我们提出了ArtCoT，展示了艺术特定任务分解和使用具体语言如何提升MLLMs在美学方面的推理能力。我们的发现为MLLMs在艺术领域提供了宝贵的见解，并可以使一系列下游应用受益，例如风格转移和艺术图像生成。代码可在https://github.com/songrise/MLLM4Art 上找到。

English

We present the first study on how Multimodal LLMs' (MLLMs) reasoning ability shall be elicited to evaluate the aesthetics of artworks. To facilitate this investigation, we construct MM-StyleBench, a novel high-quality dataset for benchmarking artistic stylization. We then develop a principled method for human preference modeling and perform a systematic correlation analysis between MLLMs' responses and human preference. Our experiments reveal an inherent hallucination issue of MLLMs in art evaluation, associated with response subjectivity. ArtCoT is proposed, demonstrating that art-specific task decomposition and the use of concrete language boost MLLMs' reasoning ability for aesthetics. Our findings offer valuable insights into MLLMs for art and can benefit a wide range of downstream applications, such as style transfer and artistic image generation. Code available at https://github.com/songrise/MLLM4Art.