AMO 取樣器：通過超出預測來增強文本呈現

摘要

在文本到圖像生成中實現文本指示與生成的圖像之間的精確對齊是一個重大挑戰，特別是在圖像中呈現書面文字方面。像Stable Diffusion 3（SD3）、Flux和AuraFlow這樣的最先進模型仍然在準確呈現文本方面遇到困難，導致拼寫錯誤或文本不一致。我們提出了一種無需訓練且計算開銷極小的方法，顯著提高了文本呈現質量。具體來說，我們引入了一種用於預訓練矯正流（RF）模型的超越取樣器，通過在學習的常微分方程（ODE）之間交替進行過度模擬和重新引入噪聲。與Euler取樣器相比，超越取樣器有效地引入了一個額外的 Langevin 動力學項，有助於糾正連續 Euler 步驟中的累積誤差，從而改善文本呈現。然而，當超越強度較高時，我們觀察到在生成的圖像上出現過度平滑的人工瑕疵。為了解決這個問題，我們提出了一種注意力調節的超越取樣器（AMO），根據它們與文本內容的注意力分數，自適應地控制每個圖像塊的超越強度。AMO 在不影響整體圖像質量或增加推理成本的情況下，在 SD3 和 Flux 上分別展示了 32.3% 和 35.9% 的文本呈現精確度提高。

English

Achieving precise alignment between textual instructions and generated images in text-to-image generation is a significant challenge, particularly in rendering written text within images. Sate-of-the-art models like Stable Diffusion 3 (SD3), Flux, and AuraFlow still struggle with accurate text depiction, resulting in misspelled or inconsistent text. We introduce a training-free method with minimal computational overhead that significantly enhances text rendering quality. Specifically, we introduce an overshooting sampler for pretrained rectified flow (RF) models, by alternating between over-simulating the learned ordinary differential equation (ODE) and reintroducing noise. Compared to the Euler sampler, the overshooting sampler effectively introduces an extra Langevin dynamics term that can help correct the compounding error from successive Euler steps and therefore improve the text rendering. However, when the overshooting strength is high, we observe over-smoothing artifacts on the generated images. To address this issue, we propose an Attention Modulated Overshooting sampler (AMO), which adaptively controls the strength of overshooting for each image patch according to their attention score with the text content. AMO demonstrates a 32.3% and 35.9% improvement in text rendering accuracy on SD3 and Flux without compromising overall image quality or increasing inference cost.

AMO 取樣器：通過超出預測來增強文本呈現

AMO Sampler: Enhancing Text Rendering with Overshooting

摘要

Summary

Support