Omegance：擴散式合成中不同粒度的單一參數

Omegance: A Single Parameter for Various Granularities in Diffusion-Based Synthesis

November 26, 2024

作者: Xinyu Hou, Zongsheng Yue, Xiaoming Li, Chen Change Loy

cs.AI

摘要

在這項工作中，我們引入了一個單一參數 omega，以有效地控制擴散式合成中的粒度。這個參數是在擴散模型反向過程的去噪步驟中加入的。我們的方法不需要重新訓練模型、架構修改或在推論期間增加額外的計算負擔，但能夠精確控制生成輸出中的細節水平。此外，可以應用具有不同 omega 值的空間遮罩或去噪時間表，以實現區域特定或時間步特定的粒度控制。從控制信號或參考圖像中的圖像組成的先前知識進一步有助於為特定對象的粒度控制創建精確的 omega 遮罩。為了突顯參數在控制微妙細節變化中的作用，這項技術被命名為 Omegance，結合了"omega"和"nuance"。我們的方法在各種圖像和視頻合成任務中展現出令人印象深刻的性能，並且適用於先進的擴散模型。程式碼可在 https://github.com/itsmag11/Omegance 找到。

English

In this work, we introduce a single parameter omega, to effectively control granularity in diffusion-based synthesis. This parameter is incorporated during the denoising steps of the diffusion model's reverse process. Our approach does not require model retraining, architectural modifications, or additional computational overhead during inference, yet enables precise control over the level of details in the generated outputs. Moreover, spatial masks or denoising schedules with varying omega values can be applied to achieve region-specific or timestep-specific granularity control. Prior knowledge of image composition from control signals or reference images further facilitates the creation of precise omega masks for granularity control on specific objects. To highlight the parameter's role in controlling subtle detail variations, the technique is named Omegance, combining "omega" and "nuance". Our method demonstrates impressive performance across various image and video synthesis tasks and is adaptable to advanced diffusion models. The code is available at https://github.com/itsmag11/Omegance.