BiGR：利用二進制潛在碼進行圖像生成和提高視覺表示能力

BiGR: Harnessing Binary Latent Codes for Image Generation and Improved Visual Representation Capabilities

October 18, 2024

作者: Shaozhe Hao, Xuantong Liu, Xianbiao Qi, Shihao Zhao, Bojia Zi, Rong Xiao, Kai Han, Kwan-Yee K. Wong

cs.AI

摘要

我們介紹了一種名為BiGR的新型條件圖像生成模型，該模型使用緊湊的二進制潛在碼進行生成式訓練，旨在增強生成和表示能力。BiGR是第一個在同一框架內統一生成和區分的條件生成模型。BiGR具有二進制分詞器、遮罩建模機制和二進制轉碼器，用於二進制碼預測。此外，我們引入了一種新穎的熵排序採樣方法，以實現有效的圖像生成。大量實驗驗證了BiGR在生成質量（以FID-50k衡量）和表示能力（通過線性探針準確度證明）方面的優越性能。此外，BiGR展示了在各種視覺任務中的零樣本泛化能力，實現了諸如圖像修補、外部補充、編輯、插值和豐富化等應用，無需進行結構修改。我們的研究結果表明，BiGR有效地統一了生成和區分任務，為該領域的進一步發展鋪平了道路。

English

We introduce BiGR, a novel conditional image generation model using compact binary latent codes for generative training, focusing on enhancing both generation and representation capabilities. BiGR is the first conditional generative model that unifies generation and discrimination within the same framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a binary transcoder for binary code prediction. Additionally, we introduce a novel entropy-ordered sampling method to enable efficient image generation. Extensive experiments validate BiGR's superior performance in generation quality, as measured by FID-50k, and representation capabilities, as evidenced by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization across various vision tasks, enabling applications such as image inpainting, outpainting, editing, interpolation, and enrichment, without the need for structural modifications. Our findings suggest that BiGR unifies generative and discriminative tasks effectively, paving the way for further advancements in the field.