Skip to main content
Back to timeline
arXivSource publication:

A correction step for flow matching: Marginal Matching reaches FID 3.45 on CIFAR-10 at 41 function evaluations

Synopsis

Flow matching models turn noise into samples along an ordinary differential equation, but the learned flow field is imperfect and the sampling trajectory drifts from the true data distribution. Marginal Matching inserts a separately trained corrector during inference that pulls samples toward the marginal distribution obtained by integrating the same flow field backward, backed by an analysis showing the correction lowers a bound on Wasserstein distance. On CIFAR-10, with OT-CFM as the base model and 10-step RK4 sampling (40 function evaluations), one correction step gives FID 3.45, against 4.34 for the uncorrected OT-CFM RK4 baseline.

Source-provided article image: Correcting Flows with Marginal Matching
arXiv

Interpretation

Flow matching sampling gains a trainable correction stage: between predictor steps, an auxiliary network pushes the current sample toward the marginal distribution of the reversed flow field instead of relying on forward integration alone. Earlier predictor-corrector schemes for flow matching were mainly training-free, for example caching previous results to approximate the ODE solution; diffusion models already had trained correctors, but these had not been carried over to flow matching. On CIFAR-10, the OT-CFM base with 10-step RK4 sampling uses 40 function evaluations and reaches FID 4.34; adding one score correction gives 41 function evaluations and FID 3.45, and three corrections give 43 function evaluations and FID 3.38.

The correction direction comes from time reversibility: integrating the pretrained flow field backward from data samples to noise yields intermediate distributions on that reverse trajectory, which serve as the correction targets. Conventional predictor-corrector pulls samples toward the forward interpolated marginals used in training, whereas this approach uses the marginals given by the reverse ODE, since only those land exactly on the data distribution in the ideal case. On CIFAR-10, comparing the two targets, score correction toward backward marginals gives FID 3.45 at 41 function evaluations, while correction toward interpolated marginals gives 3.46; using all correction steps for interpolated marginals steers sampling off course and degrades results.

The correction function has two implementations: a time-dependent score model, and a time-dependent classifier that separates forward from backward marginals and whose gradient pushes samples toward the backward side. The score model never sees samples from the forward marginals during training, while the classifier explicitly uses samples from both sides and relies on adversarial training and gradient alignment to keep its gradient stable and pointed at the backward marginals. On CIFAR-10, score correction gives FID 3.45 for OT-CFM and 3.47 for I-CFM at 41 function evaluations; classifier correction gives 3.57 for OT-CFM and 3.77 for I-CFM at 42 function evaluations, making it more sensitive to the base model.

Perspective

The demonstrated scope is image generation: unconditional generation on CIFAR-10, and score correction on ImageNet-64 with a pretrained OT-CFM base; the corrector is a UNet with about half the flow model's parameters and far fewer training iterations. The authors propose joint training of the flow and correction models as a future direction.

The method assumes the flow field is invertible, and the authors state this assumption cannot be verified in practice, only tested empirically; the theoretical analysis assumes the per-step error reduction is known, while in practice only the score correction has a bound and the classifier-based correction has no corresponding theoretical bound.