DreamCache：基于特征缓存的无微调轻量化个性化图像生成

DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching

November 26, 2024

作者: Emanuele Aiello, Umberto Michieli, Diego Valsesia, Mete Ozay, Enrico Magli

cs.AI

摘要

个性化图像生成需要文本到图像生成模型，这些模型捕捉参考主题的核心特征，以便在不同上下文中实现可控生成。现有方法面临挑战，因为训练要求复杂、推断成本高、灵活性有限，或者以上述问题的组合。在本文中，我们介绍了DreamCache，这是一种可扩展的方法，用于高效且高质量的个性化图像生成。通过缓存少量来自部分层和预训练扩散去噪器的单个时间步的参考图像特征，DreamCache通过轻量级、经过训练的调节适配器实现对生成图像特征的动态调制。DreamCache实现了最先进的图像和文本对齐，利用数量级更少的额外参数，并且比现有模型更具计算效率和多功能性。

English

Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models.