从元素到设计：一种分层方法用于自动图形设计 组合

From Elements to Design: A Layered Approach for Automatic Graphic Design Composition

December 27, 2024

作者: Jiawei Lin, Shizhao Sun, Danqing Huang, Ting Liu, Ji Li, Jiang Bian

cs.AI

摘要

在这项工作中，我们研究了从多模态图形元素中进行自动设计构成。尽管最近的研究已经为图形设计开发了各种生成模型，但它们通常面临以下限制：它们只关注特定子任务，远未达到设计构成任务；它们在生成过程中未考虑图形设计的层次信息。为了解决这些问题，我们将分层设计原则引入大型多模态模型（LMMs），并提出了一种新方法，称为LaDeCo，以完成这一具有挑战性的任务。 具体而言，LaDeCo首先为给定元素集执行层规划，根据其内容将输入元素划分为不同的语义层。基于规划结果，它随后以逐层方式预测控制设计构成的元素属性，并将先前生成的层的渲染图像包含在上下文中。通过这种富有洞察力的设计，LaDeCo将困难的任务分解为更小的可管理步骤，使生成过程更加顺畅和清晰。实验结果证明了LaDeCo在设计构成中的有效性。此外，我们展示了LaDeCo在图形设计中实现一些有趣应用的能力，如分辨率调整、元素填充、设计变体等。此外，它甚至在一些设计子任务中表现优于专门模型，而无需进行任何特定任务的训练。

English

In this work, we investigate automatic design composition from multimodal graphic elements. Although recent studies have developed various generative models for graphic design, they usually face the following limitations: they only focus on certain subtasks and are far from achieving the design composition task; they do not consider the hierarchical information of graphic designs during the generation process. To tackle these issues, we introduce the layered design principle into Large Multimodal Models (LMMs) and propose a novel approach, called LaDeCo, to accomplish this challenging task. Specifically, LaDeCo first performs layer planning for a given element set, dividing the input elements into different semantic layers according to their contents. Based on the planning results, it subsequently predicts element attributes that control the design composition in a layer-wise manner, and includes the rendered image of previously generated layers into the context. With this insightful design, LaDeCo decomposes the difficult task into smaller manageable steps, making the generation process smoother and clearer. The experimental results demonstrate the effectiveness of LaDeCo in design composition. Furthermore, we show that LaDeCo enables some interesting applications in graphic design, such as resolution adjustment, element filling, design variation, etc. In addition, it even outperforms the specialized models in some design subtasks without any task-specific training.