归纳矩匹配

Inductive Moment Matching

March 10, 2025

作者: Linqi Zhou, Stefano Ermon, Jiaming Song

cs.AI

摘要

扩散模型和流匹配方法虽能生成高质量样本，但在推理时速度较慢，且将其蒸馏为少步模型常导致不稳定性和大量调参需求。为解决这些权衡问题，我们提出了归纳矩匹配（IMM），这是一种专为一步或少数步采样设计的新型生成模型，采用单阶段训练流程。与蒸馏不同，IMM无需预训练初始化及双网络优化；相较于一致性模型，IMM确保了分布层面的收敛性，并在多种超参数及标准模型架构下保持稳定。在ImageNet-256x256数据集上，IMM仅用8步推理便以1.99的FID超越了扩散模型，并在CIFAR-10上实现了从零训练模型的最优两步FID，达到1.98，创下新纪录。

English

Diffusion models and Flow Matching generate high-quality samples but are slow at inference, and distilling them into few-step models often leads to instability and extensive tuning. To resolve these trade-offs, we propose Inductive Moment Matching (IMM), a new class of generative models for one- or few-step sampling with a single-stage training procedure. Unlike distillation, IMM does not require pre-training initialization and optimization of two networks; and unlike Consistency Models, IMM guarantees distribution-level convergence and remains stable under various hyperparameters and standard model architectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FID using only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98 on CIFAR-10 for a model trained from scratch.