DreamCache：通過特徵緩存實現無微調輕量化個性化圖像生成

DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching

November 26, 2024

作者: Emanuele Aiello, Umberto Michieli, Diego Valsesia, Mete Ozay, Enrico Magli

cs.AI

摘要

個性化圖像生成需要文本到圖像生成模型，這些模型能夠捕捉參考主題的核心特徵，以實現在不同情境下的可控生成。現有方法面臨著複雜的訓練需求、高推論成本、有限的靈活性，或這些問題的組合所帶來的挑戰。在本文中，我們介紹了DreamCache，一種可擴展的方法，用於高效且高質量的個性化圖像生成。通過從部分層的少量參考圖像特徵和預先訓練的擴散去噪器的單個時間步長中緩存，DreamCache實現了通過輕量級、訓練的條件適配器對生成的圖像特徵進行動態調節。DreamCache實現了最先進的圖像和文本對齊，利用了數量級更少的額外參數，並且比現有模型更具計算效率和多功能性。

English

Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models.