K-LoRA：实现无需训练的任意主题与风格LoRAs融合

K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs

February 25, 2025

作者: Ziheng Ouyang, Zhen Li, Qibin Hou

cs.AI

摘要

近期研究探索了将不同LoRA（低秩适应）模型结合以共同生成学习到的风格与内容。然而，现有方法要么无法有效同时保留原始主体与风格，要么需要额外训练。本文主张，LoRA的内在特性能够有效指导扩散模型在学习主体与风格之间进行融合。基于这一洞见，我们提出了K-LoRA，一种简单却高效的无训练LoRA融合方法。在每一注意力层中，K-LoRA比较待融合各LoRA中的Top-K元素，决定选择哪个LoRA以实现最优融合。这一选择机制确保了融合过程中主体与风格最具代表性的特征得以保留，有效平衡了二者的贡献。实验结果表明，所提方法成功整合了原始LoRA学习到的主体与风格信息，在定性与定量结果上均超越了基于训练的最先进方法。

English

Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.