表示分布匹配（RDM）用于一步视觉生成

Lan Feng

Wuyang Li

Éloi Zablocki

Matthieu Cord

Alexandre Alahi

EPFL, Switzerland

Valeo.ai, France

Sorbonne Université, France

Abstract

We elucidate the design space of Representation Distribution Matching (RDM), our name for the paradigm that trains a one-step image generator by matching generated and reference feature distributions under frozen pretrained encoders. We identify two design axes, how the distributions are compared and the representations they are compared in, and controlled studies along them yield three findings. First, the classical MMD, which could not train convincing generators a decade ago, becomes a strong and scalable objective once estimated right. Second, the generated batch is then the operative variable, with an optimum above 2048, far beyond customary batch sizes. Third, any single representation can be gamed, driven below the real score while images stay visibly fake, so we match against a balanced battery of encoders and evaluate with , a Sliced-Wasserstein distance over 14 encoders that is independent of the training loss and resists gaming. Combining the preferred choices yields improved RDM (iRDM): it sets the one-step state of the art on ImageNet at 1.30, corroborated by PickScore, a human-preference proxy our objective never optimizes, which prefers it over the prior best one-step generator on 71.2% of matched samples. The same recipe post-trains the four-step FLUX.2 [klein] into a one-step generator, surpassing the four-step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU-hours. Project page: https://alan-lanfeng.github.io/rdm/.

1 Introduction

Generative modeling is fundamentally distribution matching: we want a generator whose output distribution matches the data, and we already judge that match by the distance between their representation distributions, the basis of FID (Heusel et al., 2017). Diffusion and flow models pursue this distributional goal only implicitly, learning to reverse a noising process so that many denoising steps, simulated at inference, carry noise onto the data (Ho et al., 2020; Song et al., 2021; Lipman et al., 2023). A recent alternative pursues it explicitly and directly, matching the two distributions in the feature space of a frozen pretrained encoder and producing an image in a single network evaluation, with no online teacher, adversary, or trajectory to simulate. We refer to this paradigm as Representation Distribution Matching (RDM).

Several recent one-step generators (Deng et al., 2026; Yang et al., 2026) can be viewed as this paradigm, differing along just two axes. The first is the comparison: which discrepancy scores the gap between the generated and real feature laws, how it is estimated from finite samples, and what reference stands in for each side. The drifting field measures pairwise kernel forces within each batch and reads its real reference off that same batch (Deng et al., 2026), while the Fréchet-distance loss keeps only the first two moments, precomputed over the full dataset (Yang et al., 2026). The second axis is the representations, by which we mean the frozen encoder feature spaces in which the two distributions are compared; here every method has settled on the same default, a few encoders under fixed weights.

Existing methods fix these choices jointly, so it is unclear which of them is responsible for quality. We vary one axis at a time, and the resulting controlled studies overturn several assumptions implicit in current practice.

Start with the comparison. The maximum mean discrepancy was dismissed a decade ago as too weak to train a competitive generator (Li et al., 2015; Dziugaite et al., 2015); it was never too weak, only badly estimated. A good estimate needs a structured feature space and enough samples on each side, and the two sides differ. The reference is fixed in advance and never moves, so we use all of it: the entire 1.28M-image training set is compressed once into a frozen Nyström reference (Chatalic et al., 2022), 4096 landmarks standing in for the attraction at a fraction of the cost (fig. 3). The generated side moves at every step and is drawn fresh, where a larger batch sharpens the estimate but buys fewer updates; the optimum lies above 2048, an order of magnitude past common practice, with gradient caching (Gao et al., 2021) absorbing the memory. Finally, on conditional tasks we match the joint law of caption and image features rather than the image marginal alone, making prompt fidelity part of the objective, which post-trains four-step FLUX.2 into a one-step model at a higher GenEval.

Now the representations. Modern pretrained encoders already provide good spaces in which to measure the distance, so the question is which space, or which combination of spaces, makes a low MMD achievable only by genuinely realistic samples. A single encoder is not enough: the generator overfits whichever one it trains against, beating the real data on that encoder’s own score while its samples stay visibly fake. The fix is to rely on none alone. We match across a diverse battery of encoders, and rather than weight them uniformly we keep them in balance by constrained optimization: a proportional Lagrangian controller (Stooke et al., 2020) upweights whichever encoder is hardest to satisfy and downweights whichever the generator is beginning to overfit. The intuition is the weakest-stave rule: just as a bucket holds water only to its shortest stave, a viewer judges an image by its most pronounced artifact (Larson and Chandler, 2010; Wang and Shang, 2006), so the encoder that still objects is the one worth heeding.

Combining the two axes gives improved RDM (iRDM), a simple but effective recipe that generates in a single step at higher quality. We measure it with our new metric , a relative Sliced-Wasserstein distance averaged over 14 pretrained encoders, with real data scaled to 1. As an evaluation metric the Sliced-Wasserstein distance is harder to game than the Fréchet distance or the MMD (Berthet et al., 2026), and since we never train against it, a gain rules out reward hacking. Post-training pMF-H FD-SIM (Lu et al., 2026; Yang et al., 2026), whose result held the previous state of the art 2.05, iRDM reaches a new one-step state of the art at 1.30, corroborated by a PickScore (Kirstain et al., 2023) win rate, a learned human-preference proxy our objective never optimizes. The recipe carries to text-to-image: applied to FLUX.2 [klein] (Black Forest Labs, 2026), a 4B four-step generator, iRDM post-trains it into a one-step model that surpasses the four-step version on GenEval, 0.826 to 0.794, and on PickScore, 22.76 to 22.58, in 90 H200 GPU-hours. We summarize our contributions as follows.

• •

  A unifying framework. We formalize distribution matching into a single paradigm, RDM, that needs no online teacher, and identify the two design axes that govern it, how the distributions are compared and the representations they are compared in. This lets us trace the quality ceiling of a method to a specific design choice rather than its headline idea.

• •

  A simple recipe at the state of the art. Varying each axis in isolation, we establish what drives quality: the right way to estimate the MMD, an exact within-batch repulsion paired with a Nyström attraction to a frozen full-data reference; large fresh generation batches; a joint image-text objective on text-to-image tasks; and a constrained optimization that keeps a diverse encoder battery in balance. These choices combine into iRDM, which reaches state-of-the-art one-step ImageNet generation at 1.30 against the real-data 1, with no online teacher, adversary, or reward model; the same recipe post-trains four-step FLUX.2 [klein] into a one-step model at a higher GenEval than the four-step base.

• •

  A metric that resists gaming. We evaluate with , a Sliced-Wasserstein distance averaged over 14 encoders and real data scoring 1 by construction, an optimal-transport metric independent of the training loss and far harder to game than any single-encoder score.

2 Related Work

One-step and few-step generation.

Diffusion and flow models (Ho et al., 2020; Song et al., 2021; Lipman et al., 2023; Karras et al., 2022; Rombach et al., 2022; Peebles and Xie, 2023) pay an inference cost per denoising step. Step reduction either distills a pretrained teacher or removes it. Distillation matches the teacher’s trajectory, score, or moments (Salimans and Ho, 2022; Liu et al., 2023; Luo et al., 2023; Yin et al., 2024b; Zhou et al., 2024; Salimans et al., 2024) or trains against an adversary (Sauer et al., 2024; Yin et al., 2024a); the teacher-free route constrains the model on its own outputs or trajectories (Song et al., 2023; Song and Dhariwal, 2024; Geng et al., 2025b; Lu and Song, 2025; Frans et al., 2025; Geng et al., 2025a; Lu et al., 2026; Geng et al., 2026; Zhou et al., 2025). RDM needs no online teacher and constrains no trajectory: it compares generated samples against a frozen reference directly.

Matching distributions in fixed feature spaces.

Casting generation as distribution matching is the GAN program (Goodfellow et al., 2014; Salimans et al., 2016); with fixed kernels it gave moment matching networks (Li et al., 2015; Dziugaite et al., 2015), adversarial kernels (Li et al., 2017; Bińkowski et al., 2018), and sliced Wasserstein generators (Deshpande et al., 2018; Wu et al., 2019). What changed since is the feature space: frozen pretrained encoders, used for perceptual losses (Johnson et al., 2016; Zhang et al., 2018), discriminator features (Sauer et al., 2021; Kumari et al., 2022), and alignment targets (Yu et al., 2025), now support direct feature-distribution matching, as the drifting field (Deng et al., 2026), the Fréchet-distance loss (Yang et al., 2026), and a concurrent Sinkhorn flow (Han et al., 2026) show. The principle runs implicitly through this lineage; our contribution is to name it, chart its two design axes, and locate prior methods within them.

3 Representation Distribution Matching and its design space

A one-step generator maps a prior to an image in a single evaluation, with output law . Given a frozen encoder that sends an image to a feature , RDM aligns the feature laws of generated and real data (fig. 2),

where is the pushforward and a distance between distributions. Constraining the output distribution rather than a per-sample trajectory makes the generator one-step by construction; the same objective post-trains a few-step sampler by treating its final output as .

Every instance of eq. 1 is fixed by two choices, the axes of this paper: the comparison, set by which discrepancy scores the feature laws, which estimator computes it from finite samples, what reference stands in for each side, and which joint law is matched under conditioning (Sections 3.1 and 3.2); and the representations, which encoders define the feature spaces and how several are weighted (Section 3.3).

Our decomposition locates prior methods on these axes and attributes each method’s ceiling to a specific choice. The Fréchet-distance loss freezes a global data-side reference, the right call, but compresses it to two moments, so matching can saturate while images stay flawed. The drifting field has a sharp pairwise estimator, but it rebuilds its reference from every batch at a cost that confines it to small batches, exactly where a distribution estimate is noisiest. Both train against a few encoders under fixed weights, which Section 3.3 shows is gameable. iRDM is the combination of the preferred choice on each axis.

3.1 The comparison axis: choosing and estimating the discrepancy

A positive definite kernel on feature space defines

which vanishes exactly when for a characteristic kernel such as the Gaussian (Gretton et al., 2012; Sriperumbudur et al., 2010). We adopt the squared MMD with this Gaussian kernel, , on the raw encoder embeddings; the bandwidth is fixed per encoder by the median heuristic and held at a single scale. What a generator optimizes is a finite-sample estimate, and the estimator sets its cost, its variance, and the blind spots it can exploit.

Write for the features of a generated batch of size . Of the three terms of eq. 2, the data term is constant in and dropped; the cross term attracts generated features toward the data; the generator term repels them from one another, the only force preventing collapse onto the densest modes. The two demands are opposite, so we estimate the terms differently,

where is the Nyström feature map and the frozen reference mean embedding it induces over the full training set, both made precise below. Every batch is scored by all encoders in the battery, and we sum over them each step with the adaptive weights of Section 3.3.

An exact repulsion, a frozen attraction.

The two terms sum over different sets and we estimate them differently. The repulsion runs only within the batch, where the exact kernel sum is cheap, so we leave it exact, one matrix per encoder. The attraction instead compares against the full training set: resampling it each step, as the standard two-sample estimator does, injects reference noise that grows as the bandwidth shrinks, so we compute it once and freeze it through a Nyström kernel mean embedding (Chatalic et al., 2022). With landmarks placed by -means on the data features and kernel matrix , makes the Nyström approximation of , and is precomputed once over all M training images and frozen. Each step pulls the batch toward this zero-variance summary at cost , negligible next to the encoder forward passes.

Why MMD, and why Nyström.

Each alternative discrepancy gives up one of these advantages: the Fréchet distance collapses each side to two moments and can saturate while samples stay flawed; sliced-Wasserstein relies on sorting within each projection, so it scores the batch only against a resampled batch rather than the full real distribution (Deshpande et al., 2018); and the drifting field is a per-particle normalized form of the same MMD gradient, steadier at small batches but reducing to the plain MMD as the batch grows, its resampled per-batch reference keeping it small-batch (Deng et al., 2026). For the attraction term, Nyström landmarks beat random Fourier features (Rahimi and Recht, 2007): the landmark basis is data-dependent, centered on real points and accurate exactly where generation happens, whereas global cosines spend capacity over an ambient space the manifold barely occupies and leave unresolved directions that a generator under optimization pressure exploits. Theory concurs, with data-dependent bases dominating once the kernel spectrum decays quickly and of order landmarks retaining the exact embedding’s rate (Yang et al., 2012; Chatalic et al., 2022). A controlled study makes both choices concrete.

A controlled study of the estimator.

Real encoders place data on a thin manifold in a high-dimensional space, and we isolate this regime with a known target: following Li and He (2025), a two-turn spiral buried in by a fixed orthonormal map, the same MLP generator trained under each objective at a matched budget while the batch sweeps , scored by anchor recall and medDist, the median distance to the curve, on which real data scores 0.033 (settings in fig. 3).

At the largest batch both MMD estimators, MMD exact and MMD Nyström, lock onto the spiral, while random features stay diffuse, sliced-Wasserstein stays loose, and drifting collapses. As the batch shrinks, MMD exact degrades as its per-batch reference thins, whereas MMD Nyström pulls toward the same frozen reference at every batch size and stays sharpest in every row; sliced-Wasserstein loses recall at the smallest batch and drifting collapses at the largest. MMD Nyström is the only method that fails nowhere.

3.2 The comparison axis: batches and conditioning

The generator side: large, fresh batches.

With the data side frozen once over the full training set, the generated distribution is the only quantity still moving, and it moves at every step, so it must be sampled fresh; estimating it from a stale buffer, as the EMA queue of Yang et al. (2026) does, biases the gradient off-policy. A fresh batch makes its size the operative variable: a larger lowers the variance of the estimate but, at a fixed compute budget, buys fewer optimizer steps, trading estimate sharpness against the number of updates. Large fresh batches are normally ruled out by memory, which gradient caching (Gao et al., 2021) removes by accumulating the exact full-batch gradient in chunks at the cost of one chunk. We sweep at a matched wall-clock budget, scaling the learning rate as (Malladi et al., 2022) so every arm sees about one epoch split into more or fewer updates (fig. 4). Quality climbs with : the trained encoder sharpens while the held-out-dominated panel barely moves, the smallest batch is noise-dominated and regresses despite far more optimizer steps, and the curve then flattens into a broad optimum. We adopt for ImageNet and the larger for the FLUX post-training; exact values are in Appendix C.

Conditional tasks: match the joint, not the marginal.

A prompted generator can satisfy the image marginal while drifting from its prompts: realism bought with alignment. We instead match the joint law. With a frozen text encoder and coupled features ,

where reference pairs couple each image with its caption and generated pairs couple each output with the prompt that produced it; the estimator is unchanged, landmarks now reference image-text pairs. Under the kernel a generated image is pulled toward reference images whose captions resemble its prompt, so prompt fidelity is part of what is matched. Post-training the four-step FLUX.2 [klein] (Black Forest Labs, 2026) into a one-step model with this objective surpasses the four-step version on GenEval (Ghosh et al., 2023) while also surpassing its PickScore (Section 4.2); the marginal alternative sacrifices alignment with no compensating quality gain (Table 2).

3.3 The representation axis: one encoder is never enough

Feature distances are also how realism is scored: FID and its descendants (Heusel et al., 2017; Bińkowski et al., 2018; Jayasumana et al., 2024; Stein et al., 2023) reduce it to the distributional gap under one pretrained encoder, read as a proxy for human judgment. The proxy is fragile. FID falls under fringe ImageNet-class features with no gain in perceived quality (Kynkäänniemi et al., 2023), and such a distance is directly optimizable: a generator can be driven below the score of real validation data while staying visibly fake (Yang et al., 2026). The question this axis turns on: is there any feature space whose distance, once minimized, yields images humans cannot tell from real?

Overfitting a single encoder.

Below-real scores have so far been shown only on weak proxies, Inception and ConvNeXt, inviting the objection that a sufficiently rich encoder, once satisfied, would force realism. We test the hardest case we can construct: DINOv2, far more semantically structured, on which the base checkpoint starts far from real, . Matching it alone, for steps, drives the distance to , essentially the real-validation floor of : by DINOv2’s account the generator is as close to real as real data. fig. 5 says otherwise. The objective repairs some classes, the lizard becomes hard to tell from a photograph, and leaves others untouched, the typewriter keeps an implausible key layout at that same floor score. The limitation is single-encoder matching itself, not the choice of encoder, and the resolution is not a better encoder but a diverse ensemble.

Constrained optimization against multiple encoders.

A single encoder gives only a pseudometric, but the combined kernel of a diverse panel is characteristic and vanishes only at the real distribution (Gretton et al., 2012; Sriperumbudur et al., 2010; Schrab et al., 2023); we therefore train against ten of the fourteen panel encoders (Appendix B), frozen backbones chosen to fail in different ways. The weighting then decides whether this diversity survives: under fixed weights the optimizer drives the aggregate down through whichever encoders are easiest. We instead pose the weighting as a constrained optimization, each encoder required to reach its real-validation floor with its weight the Lagrange multiplier, set by proportional control under a satisfaction gate, the proportional term of the PID-Lagrangian scheme of Stooke et al. (2020). An encoder’s excess sets its weight: those at or below their floor drop out, while the violators share a fixed budget through a softmax, , so the representations farthest from real are weighted most; when all are satisfied the weights vanish, a natural anti-overfitting terminal state.

Scaling the multi-representation metric.

Evaluation needs the same protection and must not collapse into the training loss. Yang et al. (2026) aggregate a per-encoder ratio over a panel, the Fréchet form ; we keep that construction but replace the Fréchet distance with the Sliced-Wasserstein (Deshpande et al., 2018; Wu et al., 2019), a proper optimal-transport distance that shares no estimator with the MMD we train against (Berthet et al., 2026). Our metric averages the per-encoder ratio over the encoders,

With , real validation data scores by construction, a floor no released generator approaches (Table 1); four of the encoders are held out from training as a generalization check. Appendix D gives a kernel-MMD counterpart MMDr14, and Section 4.1 validates against PickScore.

Putting it together: iRDM.

Together these choices define iRDM: an exact within-batch repulsion with a Nyström attraction to a reference frozen once over the full data, large fresh generation batches, the joint image-text law on conditional tasks, and a diverse encoder battery balanced by constrained optimization. The reference is precomputed once per encoder; each step then draws a fresh batch, generates in a single evaluation, encodes it under the training encoders with gradient caching, and sums the per-encoder losses of eq. 3 under the proportional Lagrangian weights. Nothing else enters the objective: no online teacher, no adversary, no trajectory.

4 Experiments

Sections 3.1 to 3.3 fixed each design choice with a controlled study in place. The experiments report what remains: the main results, one-step ImageNet generation and text-to-image post-training, and the ablations the studies did not cover.

4.1 One-step ImageNet generation

Setup.

On ImageNet-256 (Deng et al., 2009), we post-train the released pMF-H FD-SIM checkpoint (Lu et al., 2026; Yang et al., 2026) for steps at learning rate and batch size over the ten training encoders of Appendix B, each a Gaussian kernel at its median-heuristic bandwidth whose attraction is taken against the full M-image ImageNet training set, compressed once into a -landmark Nyström reference. The ten encoders are kept in balance by the proportional Lagrangian controller of Section 3.3 with a satisfaction gate over a fixed budget , each encoder’s real floor computed on the ImageNet validation set. Evaluation uses two off-objective measures. is the Sliced-Wasserstein ratio averaged over the -encoder panel, four encoders held out from training, estimated from samples per set with projections. PickScore (Kirstain et al., 2023) is a learned human-preference model scored against the class prompt; against the pMF-H FD-SIM start we render class-conditional latents under matched noise with both models and report the paired mean and win rate.

Distributional quality.

Table 1 places released ImageNet-256 generators on ; none approaches the real floor of , the strongest reaching about . iRDM sets the state of the art at , below every released generator, and is the best entry on nine of the fourteen encoders and on the aggregate. It cedes five: Inception, ConvNeXt, and MAE to the FD-loss model, which scores below real there by gaming a single space, DreamSim to that same model by a hair, and the held-out FLUX VAE to MAR-H. Appendix D reports the same field under MMDr14, a kernel-MMD panel, which broadly agrees, with some reordering among the mid-field models.

Human preference.

An off-objective check agrees. PickScore (Kirstain et al., 2023), a learned human-preference model we never train against, prefers our converged checkpoint to its pMF-H FD-SIM start on of matched pairs (, paired ) and to the recent RAE-XL (Zheng et al., 2025) and REPA-E SiT-XL (Leng et al., 2025) on and of classes (Figure 6); it even prefers our samples to held-out real photographs on , to our knowledge the first one-step generator to pass the real-image PickScore.

4.2 Text-to-image post-training

Setup.

We post-train FLUX.2 [klein] (Black Forest Labs, 2026) from its four-step checkpoint into a one-step model with the joint image-text objective of Section 3.2, at batch size and learning rate for steps, about H200 GPU-hours, under the encoder battery and constrained-optimization weighting of Section 4.1. The matching reference is collected once from the four-step teacher and then frozen, so the post-training queries no online teacher: a curated set of about K teacher generations, PickScore-ranked COCO renderings (Lin et al., 2014) together with detector-verified GenEval-correct samples, compressed once into a Nyström reference and detailed in Appendix E.1. We evaluate with GenEval (Ghosh et al., 2023) under its standard protocol and PickScore (Kirstain et al., 2023) on COCO validation prompts, and compare against a DMD2 (Yin et al., 2024a) one-step distillation of the same four-step teacher (Appendix E.2).

Results.

The one-step model surpasses its four-step start on GenEval overall, against , with the per-category breakdown in Table 2: it matches the four-step version on single-object prompts, exceeds it on two-object, colors, position, and attribute binding, and trails only on counting. On PickScore it reaches 22.76, also above the four-step version’s . The DMD2 baseline reaches overall GenEval and PickScore, also listed in Table 2; Figure 1(c) traces both metrics over post-training compute.

Joint versus marginal.

The joint coupling carries the gain. A marginal variant that drops the caption from the feature, matching the image marginal alone with no SigLIP text concatenation, trails the joint model overall in Table 2, against , and the gap concentrates on the categories that demand image-text alignment, two-object ( against ) and attribute binding ( against ), while single-object, which depends little on coupling, is essentially unchanged. Matching the joint law rather than the image marginal is what makes prompt fidelity part of the objective.

4.3 Constrained optimization versus uniform weighting

Setup.

We isolate the gated proportional Lagrangian controller of Section 3.3 against uniform weighting: both warm-start from pMF-H and train for steps under one recipe, with only the per-encoder allocation differing. The start is bimodal, the classic encoders already at or below their floor while the modern ones sit far from real, so the aggregate of is set by the violators.

Results.

The gated controller pours the budget onto the worst encoder, PE-Core, while gating out the three already at their floor: it edges uniform on the mean, against , while decisively improving the worst case, against from a start of , nearly twice the cut uniform manages (Table 3). Reallocating toward the largest violation is what the controller targets; we make no claim here about perceptual quality, which this aggregate does not directly measure.

4.4 The training distance

Setup.

Holding the rest of the recipe fixed, we flip only the per-step distance across six fine-tuning losses, each warm-started from the same pMF-H checkpoint and fine-tuned against a single DINOv2 cls encoder for optimizer steps, sharing AdamW at learning rate and a generation batch of . The three kernel arms use an RBF kernel at bandwidth : mmdx is the biased with an exact within-batch term and a Nyström cross-term over landmarks, mmd_exact replaces that cross-term with the exact full generated-to-real pairwise mean, and mmd_rff matches a frozen -dimensional random-Fourier-feature mean; fd is the Fréchet (Gaussian-moment) distance and sw a Sliced-Wasserstein loss with projections. The drifting arm is a faithful port of the published coupled force field across radii , with in-batch generated negatives and real positives on the same features, run time-matched to the other arms (about steps at its generation batch of ); we sweep its learning rate and report the gentlest, , since its native , tuned for from-scratch training, regresses the warm-start. We score every arm and the untrained baseline with two neutral third-party distances on the same features, a Sliced-Wasserstein ratio from samples per set with projections and an RFF-MMD ratio from samples with random Fourier features, so each arm is read on at least one distance it did not train on; the sw and mmd_rff arms, which each optimize one of the two eval distances, are judged on the other (table 4).

Results.

One ranking holds across both, mmdx mmd_rff mmd_exact fd sw drifting, well above the untrained baseline: the three kernel-MMD estimators fill the top, moment matching follows, the Sliced-Wasserstein loss next, and a faithful port of the drifting force field is weakest even at the best of a learning-rate sweep. As an objective the Nyström MMD moves the feature distribution closest to real, while optimal transport, an excellent judge, is among the least effective losses. Two controls confirm the reading: the exact full-pairwise MMD does not beat its Nyström approximation, the low-rank cross-term being a smoother gradient, and training on a distance buys no advantage on that same distance, the Sliced-Wasserstein arm being beaten on the Sliced-Wasserstein eval by every kernel-MMD arm and even by moment matching. This is why iRDM trains with the MMD-Nyström signal yet is evaluated with the independent Sliced-Wasserstein distance.

5 Conclusion

We have treated representation distribution matching, the principle behind a recent line of teacher-free one-step generators, as a design space rather than a collection of methods. Two axes fix every instance, how the generated and real feature distributions are compared and which representations they are compared in, and varying one at a time turns each into a preferred design with a mechanism behind it. On the comparison axis the classical MMD becomes a strong objective once estimated right, an exact within-batch repulsion paired with a Nyström attraction toward a reference frozen once in advance, fed by large fresh generation batches and, on conditional tasks, by matching the joint image-text law rather than the image marginal. On the representation axis no single encoder is enough, since any one can be driven below the real score while samples stay visibly fake, so we match against a diverse battery of encoders held in balance by constrained optimization. Combining these choices gives iRDM, which sets the one-step state of the art on ImageNet at and post-trains the four-step FLUX.2 [klein] into a one-step model that surpasses it on GenEval and PickScore, and we report the remaining gap with , a Sliced-Wasserstein distance over the panel that shares no machinery with the training loss.

A gap to real remains: at against a floor of , the best one-step generator is still measurably short of a fresh real draw, and narrowing it is the natural next target. The design space leaves room to do so, through multi-scale kernels, learned or task-specific encoder panels, and richer conditional couplings, and the same recipe, a frozen reference matched by a single network evaluation, should transfer to modalities beyond images wherever a pretrained encoder supplies the feature space.

Acknowledgments

We thank Jiawei Yang for helpful discussions. The project was partially funded by Valeo.

References

• Q. Berthet, Y. Wu, C. Crepy, R. Elie, K. Greff, and M. E. Sander (2026) MIND: monge inception distance for generative models evaluation. CoRR abs/2605.06797. Cited by: §1, §3.3.
• M. Bińkowski, D. J. Sutherland, M. Arbel, and A. Gretton (2018) Demystifying MMD GANs. In International Conference on Learning Representations, Cited by: Appendix A, §2, §3.3.
• Black Forest Labs (2024) FLUX.1. Note: https://github.com/black-forest-labs/fluxOfficial inference repository for FLUX.1 models Cited by: Table 5.
• Black Forest Labs (2026) FLUX.2 [klein]: towards interactive visual intelligence. Note: https://bfl.ai/blog/flux2-klein-towards-interactive-visual-intelligenceModel weights: https://huggingface.co/collections/black-forest-labs/flux2 Cited by: Appendix A, §1, §3.2, §4.2.
• K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine (2024) Training diffusion models with reinforcement learning. In International Conference on Learning Representations, Cited by: Appendix A.
• D. Bolya, P. Huang, P. Sun, J. H. Cho, A. Madotto, C. Wei, T. Ma, J. Zhi, J. Rajasegaran, H. Rasheed, J. Wang, M. Monteiro, H. Xu, S. Dong, N. Ravi, D. Li, P. Dollár, and C. Feichtenhofer (2025) Perception encoder: the best visual embeddings are not at the output of the network. In Advances in Neural Information Processing Systems, Cited by: Table 5.
• A. Chatalic, N. Schreuder, L. Rosasco, and A. Rudi (2022) Nyström kernel mean embeddings. In International Conference on Machine Learning, Cited by: Appendix A, §1, §3.1, §3.1.
• T. Coste, U. Anwar, R. Kirk, and D. Krueger (2024) Reward model ensembles help mitigate overoptimization. In International Conference on Learning Representations, Cited by: Appendix A.
• J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei (2009) ImageNet: a large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §4.1.
• M. Deng, H. Li, T. Li, Y. Du, and K. He (2026) Generative modeling via drifting. Note: arXiv preprint arXiv:2602.04770 Cited by: Appendix A, Table 7, §1, §2, §3.1.
• I. Deshpande, Z. Zhang, and A. G. Schwing (2018) Generative modeling using the sliced Wasserstein distance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §2, §3.1, §3.3.
• G. K. Dziugaite, D. M. Roy, and Z. Ghahramani (2015) Training generative neural networks via maximum mean discrepancy optimization. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, pp. 258–267. Cited by: §1, §2.
• A. Falahati, E. Creager, G. Kamath, and S. Mohapatra (2026) DriftXpress: faster drifting models via projected RKHS fields. Note: arXiv preprint arXiv:2605.12183 Cited by: Appendix A.
• D. Fan, S. Tong, J. Zhu, K. Sinha, Z. Liu, X. Chen, M. Rabbat, N. Ballas, Y. LeCun, A. Bar, and S. Xie (2025) Scaling language-free visual representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Table 5.
• E. Fini, M. Shukor, X. Li, P. Dufter, M. Klein, D. Haldimann, S. Aitharaju, V. G. Turrisi da Costa, L. Béthune, Z. Gan, A. T. Toshev, M. Eichner, M. Nabi, Y. Yang, J. M. Susskind, and A. El-Nouby (2025) Multimodal autoregressive pre-training of large vision encoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• K. Frans, D. Hafner, S. Levine, and P. Abbeel (2025) One step diffusion via shortcut models. In International Conference on Learning Representations, Cited by: §2.
• S. Fu, N. Tamir, S. Sundaram, L. Chai, R. Zhang, T. Dekel, and P. Isola (2023) DreamSim: learning new dimensions of human visual similarity using synthetic data. In Advances in Neural Information Processing Systems, Cited by: Table 5.
• L. Gao, J. Schulman, and J. Hilton (2023a) Scaling laws for reward model overoptimization. In International Conference on Machine Learning, Cited by: Appendix A.
• L. Gao, Y. Zhang, J. Han, and J. Callan (2021) Scaling deep contrastive learning batch size under memory limited setup. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pp. 316–321. External Links: Document, Link Cited by: §1, §3.2.
• S. Gao, P. Zhou, M. Cheng, and S. Yan (2023b) MDTv2: masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Appendix A, Table 7.
• Z. Geng, M. Deng, X. Bai, J. Z. Kolter, and K. He (2025a) Mean flows for one-step generative modeling. In Advances in Neural Information Processing Systems, Cited by: §2.
• Z. Geng, Y. Lu, Z. Wu, E. Shechtman, J. Z. Kolter, and K. He (2026) Improved mean flows: on the challenges of fastforward generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Appendix A, Table 7, §2.
• Z. Geng, A. Pokle, W. Luo, J. Lin, and Z. Kolter (2025b) Consistency models made easy. In International Conference on Learning Representations, Cited by: §2.
• D. Ghosh, H. Hajishirzi, and L. Schmidt (2023) GenEval: an object-focused framework for evaluating text-to-image alignment. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §E.1, §3.2, §4.2.
• I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. C. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in Neural Information Processing Systems, Cited by: §2.
• A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola (2012) A kernel two-sample test. Journal of Machine Learning Research 13, pp. 723–773. Cited by: §3.1, §3.3.
• J. Han, P. Li, Q. Guo, R. Xu, S. Ermon, and E. J. Candès (2026) One-step generative modeling via Wasserstein gradient flows. Note: arXiv preprint arXiv:2605.11755 Cited by: §2.
• K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick (2022) Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• G. Heinrich, M. Ranzinger, H. Yin, Y. Lu, J. Kautz, A. Tao, B. Catanzaro, and P. Molchanov (2025) RADIOv2.5: improved baselines for agglomerative vision foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §1, §3.3.
• J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, Cited by: §1, §2.
• S. Jayasumana, S. Ramalingam, A. Veit, D. Glasner, A. Chakrabarti, and S. Kumar (2024) Rethinking FID: towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9307–9315. Cited by: Appendix A, §3.3.
• J. Johnson, A. Alahi, and L. Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision, Cited by: §2.
• T. Karras, M. Aittala, T. Aila, and S. Laine (2022) Elucidating the design space of diffusion-based generative models. In Advances in Neural Information Processing Systems, Cited by: §2.
• Y. Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy (2023) Pick-a-Pic: an open dataset of user preferences for text-to-image generation. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §E.1, §1, §4.1, §4.1, §4.2.
• N. Kumari, R. Zhang, E. Shechtman, and J. Zhu (2022) Ensembling off-the-shelf models for GAN training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.
• T. Kynkäänniemi, T. Karras, M. Aittala, T. Aila, and J. Lehtinen (2023) The role of ImageNet classes in Fréchet inception distance. In International Conference on Learning Representations, Cited by: Appendix A, §3.3.
• T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila (2019) Improved precision and recall metric for assessing generative models. In Advances in Neural Information Processing Systems, Cited by: Appendix A.
• E. C. Larson and D. M. Chandler (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging 19 (1), pp. 011006. Cited by: §1.
• X. Leng, J. Singh, Y. Hou, Z. Xing, S. Xie, and L. Zheng (2025) REPA-E: unlocking VAE for end-to-end tuning with latent diffusion transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Table 7, §4.1.
• C. Li, W. Chang, Y. Cheng, Y. Yang, and B. Póczos (2017) MMD GAN: towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems, Cited by: §2.
• T. Li and K. He (2025) Back to basics: let denoising generative models denoise. Note: arXiv preprint arXiv:2511.13720 Cited by: Appendix A, Table 7, §3.1.
• T. Li, Y. Tian, H. Li, M. Deng, and K. He (2024) Autoregressive image generation without vector quantization. In Advances in Neural Information Processing Systems, Cited by: Appendix A, Table 7.
• Y. Li, K. Swersky, and R. Zemel (2015) Generative moment matching networks. In International Conference on Machine Learning, Cited by: §1, §2.
• T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick (2014) Microsoft COCO: common objects in context. In European Conference on Computer Vision, Cited by: §E.1, §4.2.
• Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2023) Flow matching for generative modeling. In International Conference on Learning Representations, Cited by: §1, §2.
• X. Liu, C. Gong, and Q. Liu (2023) Flow straight and fast: learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, Cited by: §2.
• Z. Liu, H. Mao, C. Wu, C. Feichtenhofer, T. Darrell, and S. Xie (2022) A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• C. Lu and Y. Song (2025) Simplifying, stabilizing and scaling continuous-time consistency models. In International Conference on Learning Representations, Cited by: §2.
• Y. Lu, S. Lu, Q. Sun, H. Zhao, Z. Jiang, X. Wang, T. Li, Z. Geng, and K. He (2026) One-step latent-free image generation with pixel mean flows. Note: arXiv preprint arXiv:2601.22158 Cited by: Appendix A, Table 7, §1, §2, §4.1.
• W. Luo, T. Hu, S. Zhang, J. Sun, Z. Li, and Z. Zhang (2023) Diff-Instruct: a universal approach for transferring knowledge from pre-trained diffusion models. In Advances in Neural Information Processing Systems, Cited by: §2.
• Z. Luo, F. Shi, Y. Ge, Y. Yang, L. Wang, and Y. Shan (2024) Open-MAGVIT2: an open-source project toward democratizing auto-regressive visual generation. Note: arXiv preprint arXiv:2409.04410 Cited by: Appendix A, Table 7.
• N. Ma, M. Goldstein, M. S. Albergo, N. M. Boffi, E. Vanden-Eijnden, and S. Xie (2024) SiT: exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, Cited by: Appendix A, Table 7.
• S. Malladi, K. Lyu, A. Panigrahi, and S. Arora (2022) On the SDEs and scaling rules for adaptive gradient algorithms. In Advances in Neural Information Processing Systems, Cited by: §3.2.
• K. Muandet, K. Fukumizu, B. Sriperumbudur, and B. Schölkopf (2017) Kernel mean embedding of distributions: a review and beyond. Foundations and Trends in Machine Learning 10 (1–2), pp. 1–141. Cited by: Appendix A.
• M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jégou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski (2024) DINOv2: learning robust visual features without supervision. Transactions on Machine Learning Research. Cited by: Table 5.
• G. Parmar, R. Zhang, and J. Zhu (2022) On aliased resizing and surprising subtleties in GAN evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Appendix A.
• W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Appendix A, Table 7, §2.
• A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever (2021) Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, Cited by: Table 5.
• A. Rahimi and B. Recht (2007) Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §3.1.
• M. Ranzinger, G. Heinrich, J. Kautz, and P. Molchanov (2024) AM-RADIO: agglomerative vision foundation model reduce all domains into one. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.
• T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen (2016) Improved techniques for training GANs. In Advances in Neural Information Processing Systems, Cited by: §2.
• T. Salimans and J. Ho (2022) Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, Cited by: §2.
• T. Salimans, T. Mensink, J. Heek, and E. Hoogeboom (2024) Multistep distillation of diffusion models via moment matching. In Advances in Neural Information Processing Systems, Cited by: §2.
• A. Sauer, K. Chitta, J. Müller, and A. Geiger (2021) Projected GANs converge faster. In Advances in Neural Information Processing Systems, Cited by: §2.
• A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach (2024) Adversarial diffusion distillation. In European Conference on Computer Vision, Cited by: Appendix A, §2.
• A. Schrab, I. Kim, M. Albert, B. Laurent, B. Guedj, and A. Gretton (2023) MMD aggregated two-sample test. Journal of Machine Learning Research 24 (194), pp. 1–81. Cited by: §3.3.
• O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. E. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jégou, P. Labatut, and P. Bojanowski (2026) DINOv3. Transactions on Machine Learning Research. Cited by: Table 5.
• J. Skalse, N. Howe, D. Krasheninnikov, and D. Krueger (2022) Defining and characterizing reward gaming. In Advances in Neural Information Processing Systems, Cited by: Appendix A.
• Y. Song, P. Dhariwal, M. Chen, and I. Sutskever (2023) Consistency models. In International Conference on Machine Learning, Cited by: §2.
• Y. Song and P. Dhariwal (2024) Improved techniques for training consistency models. In International Conference on Learning Representations, Cited by: §2.
• Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, Cited by: §1, §2.
• B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. G. Lanckriet (2010) Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research 11, pp. 1517–1561. Cited by: §3.1, §3.3.
• G. Stein, J. C. Cresswell, R. Hosseinzadeh, Y. Sui, B. Ross, V. Villecroze, Z. Liu, A. L. Caterini, E. Taylor, and G. Loaiza-Ganem (2023) Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: Appendix A, §3.3.
• A. Stooke, J. Achiam, and P. Abbeel (2020) Responsive safety in reinforcement learning by PID lagrangian methods. In International Conference on Machine Learning, Cited by: Appendix A, §1, §3.3.
• M. Strathern (1997) ‘Improving ratings’: audit in the British University system. European Review 5 (3), pp. 305–321. Cited by: Appendix A.
• C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna (2016) Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cited by: Table 5.
• K. Tian, Y. Jiang, Z. Yuan, B. Peng, and L. Wang (2024) Visual autoregressive modeling: scalable image generation via next-scale prediction. In Advances in Neural Information Processing Systems, Cited by: Appendix A, Table 7.
• M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdulmohsin, N. Parthasarathy, T. Evans, L. Beyer, Y. Xia, B. Mustafa, O. Hénaff, J. Harmsen, A. Steiner, and X. Zhai (2025) SigLIP 2: multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786. Cited by: Table 5, §E.1.
• B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik (2024) Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Appendix A.
• S. Wang, Z. Tian, W. Huang, and L. Wang (2026) DDT: decoupled diffusion transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Appendix A, Table 7.
• Z. Wang and X. Shang (2006) Spatial pooling strategies for perceptual image quality assessment. In IEEE International Conference on Image Processing, pp. 2945–2948. Cited by: §1.
• G. Wu, S. Zhang, R. Shi, S. Gao, Z. Chen, L. Wang, Z. Chen, H. Gao, Y. Tang, J. Yang, M. Cheng, and X. Li (2025) Representation entanglement for generation: training diffusion transformers is much easier than you think. In Advances in Neural Information Processing Systems, Cited by: Appendix A, Table 7.
• J. Wu, Z. Huang, D. Acharya, W. Li, J. Thoma, D. P. Paudel, and L. Van Gool (2019) Sliced Wasserstein generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2, §3.3.
• J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong (2023) ImageReward: learning and evaluating human preferences for text-to-image generation. In Advances in Neural Information Processing Systems, Cited by: Appendix A.
• J. Yang, Z. Geng, X. Ju, Y. Tian, and Y. Wang (2026) Representation fréchet loss for visual generation. Note: arXiv preprint arXiv:2604.28190 Cited by: Appendix A, Appendix A, Table 7, Figure 8, Appendix G, §1, §1, §2, §3.2, §3.3, §3.3, §4.1.
• T. Yang, Y. Li, M. Mahdavi, R. Jin, and Z. Zhou (2012) Nyström method vs random fourier features: a theoretical and empirical comparison. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §3.1.
• J. Yao, B. Yang, and X. Wang (2025) Reconstruction vs. generation: taming optimization dilemma in latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: Appendix A, Table 7.
• T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman (2024a) Improved distribution matching distillation for fast image synthesis. In Advances in Neural Information Processing Systems, Cited by: Appendix A, §E.2, Table 8, §2, §4.2, Table 2.
• T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, and T. Park (2024b) One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.
• S. Yu, S. Kwak, H. Jang, J. Jeong, J. Huang, J. Shin, and S. Xie (2025) Representation alignment for generation: training diffusion transformers is easier than you think. In International Conference on Learning Representations, Cited by: Appendix A, Table 7, §2.
• X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer (2023) Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Cited by: Table 5.
• R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang (2018) The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §2.
• B. Zheng, N. Ma, S. Tong, and S. Xie (2025) Diffusion transformers with representation autoencoders. arXiv preprint arXiv:2510.11690. Cited by: Appendix A, Table 7, §4.1.
• L. Zhou, S. Ermon, and J. Song (2025) Inductive moment matching. In International Conference on Machine Learning, Cited by: §2.
• M. Zhou, H. Zheng, Z. Wang, M. Yin, and H. Huang (2024) Score identity distillation: exponentially fast distillation of pretrained diffusion models for one-step generation. In International Conference on Machine Learning, Cited by: §2.

Appendix A Extended related work

Scalable kernel estimators.

Random Fourier features linearize kernel sums with data-independent bases [Rahimi and Recht, 2007]; Nyström methods use data-dependent landmarks instead and dominate whenever the kernel spectrum decays quickly [Yang et al., 2012]. For the kernel mean embedding, the single point in the RKHS that summarizes a distribution [Muandet et al., 2017], Nyström compression retains the full estimation rate of the exact embedding with far fewer landmarks than data points [Chatalic et al., 2022]. Concurrently, DriftXpress accelerates drifting models by projecting the kernel field onto a low-rank RKHS with landmark approximations [Falahati et al., 2026]. Our use differs in target and in symmetry: DriftXpress approximates the full per-batch drifting update for speed, while we compress only the stationary data side, into a frozen global attraction target over 1.28M images, to remove reference noise, and keep the moving within-batch repulsion exact.

Metric gaming and multi-encoder evaluation.

Single-encoder distances such as FID [Heusel et al., 2017], KID [Bińkowski et al., 2018], CMMD [Jayasumana et al., 2024], and feature-space precision and recall [Kynkäänniemi et al., 2019] inherit the blind spots of their encoder: the scores move with resizing details [Parmar et al., 2022], fall through fringe ImageNet-class features with no quality gain [Kynkäänniemi et al., 2023], and re-rank models when the encoder is swapped [Stein et al., 2023]. Yang et al. [2026] push this to the limit, driving a trained generator below the score of the real validation set; we show the failure is single-encoder matching itself rather than any weak encoder. When the proxy is learned, the same phenomenon is studied as reward hacking [Strathern, 1997, Skalse et al., 2022, Gao et al., 2023a], ensembling the proxies mitigates it [Coste et al., 2024], and Lagrangian methods control it in constrained reinforcement learning [Stooke et al., 2020]. In our setting every proxy is frozen, so gaming pressure concentrates on the encoder weighting, which is exactly what the proportional Lagrangian controller regulates; evaluation aggregates 14 encoders, 4 held out from training, into , a Sliced-Wasserstein distance independent of the loss.

Post-training text-to-image models.

Few-step text-to-image systems are typically distilled from a multi-step teacher, adversarially [Sauer et al., 2024] or by score-based distribution matching [Yin et al., 2024a], and then steered toward human taste by optimizing learned preference rewards. The reward models are trained from human choices [Xu et al., 2023, Kirstain et al., 2023] and optimized either by policy gradients [Black et al., 2024] or by direct preference objectives [Wallace et al., 2024]; all inherit the gameability of a single learned scorer, the same axis our multi-encoder battery addresses for frozen proxies. For our text-to-image result the four-step FLUX.2 [klein] [Black Forest Labs, 2026] generates a reference set in advance, and the joint image-text objective of Section 3.2 matches the one-step model against it with no online teacher and no reward model, evaluated by GenEval [Ghosh et al., 2023] and PickScore.

The evaluation landscape.

The generators placed on in Table 1 span the current families: latent diffusion transformers [Peebles and Xie, 2023, Ma et al., 2024, Gao et al., 2023b, Wang et al., 2026, Yao et al., 2025], representation-aligned variants that inject external encoder features during training [Yu et al., 2025, Wu et al., 2025, Zheng et al., 2025], autoregressive and masked-token models [Tian et al., 2024, Li et al., 2024, Luo et al., 2024], pixel-space transformers [Li and He, 2025], and the one-step MeanFlow, drifting, and FD-loss lines [Lu et al., 2026, Geng et al., 2026, Deng et al., 2026, Yang et al., 2026]. The models that use an external representation encoder in training, the starred rows of the table, populate its strongest entries, consistent with the premise that representation supervision is the operative ingredient; RDM makes that ingredient explicit and studies it in isolation.

Appendix B Encoder panel

MMDr14 takes the arithmetic mean over the encoders of table 5, each frozen at its released weights and read out as a single pooled image embedding at the listed input resolution, with no feature normalization. The panel deliberately spans training paradigms, supervised classification, self-supervised distillation and masked reconstruction, language supervision, multi-teacher agglomeration, multimodal autoregression, human similarity tuning, and a generative autoencoder, so the representations fail in different ways. Ten supervise training; the four held out for evaluation only are DINOv2, SigLIP (v1), C-RADIOv3-L, and the FLUX VAE.

Appendix C Batch-size sweep

Table 6 tabulates the sweep plotted in fig. 4.

Appendix D Kernel-MMD evaluation

Table 7 reports MMDr14, the training-aligned kernel-MMD cross-check of our primary metric, over the full released field on the -encoder panel. Each entry is a per-encoder RFF-MMD ratio against real training data, with real validation scoring by construction, and MMDr14 is their arithmetic mean over the 14 encoders. The ordering broadly agrees with (Table 1), with some reordering among the mid-field models; because the loss is itself a kernel MMD, a single encoder can be pushed below the real floor here, Inception at for the FD-SIM model, which the optimal-transport and the held-out split resist.

Appendix E Text-to-image post-training details

E.1 Reference curation

The text-to-image objective of Section 4.2 matches the one-step model against a reference collected once from the four-step FLUX.2 [klein] teacher and then frozen, so the teacher is never queried during post-training. The reference concatenates two independently curated blocks of teacher generations, a perception block and a composition block, roughly K image-caption pairs in all, each image kept with the caption that produced it for the joint kernel.

Perception block.

For each of the COCO train2014 images [Lin et al., 2014], one caption apiece, the four-step teacher draws candidates, which PickScore [Kirstain et al., 2023] ranks; we keep the three highest per caption, giving pairs at full coverage. Selecting three of twenty-four oversampled draws anchors the reference on high-quality renderings of natural captions, supplying the perceptual side of the match.

Composition block.

To pull the model toward verified-correct composition rather than the teacher’s average, which fails a large fraction of the harder compositional prompts, we keep only teacher generations a detector certifies as correct. For the GenEval [Ghosh et al., 2023] prompts the teacher is sampled at seeds per prompt, topped up where a prompt has fewer than correct, and every generation is scored by the standard GenEval Mask2Former detector: a sample passes only when all prompt objects are present at the required count, color, and spatial relation. Capping at correct per prompt yields verified images covering of the prompts; two position prompts admit no correct teacher sample even at seeds. Measured per seed over this pool, the teacher’s correctness ranges from on single-object prompts down to on attribute binding and on position, so the filter most reshapes exactly the binding and spatial prompts on which the one-step model later improves.

Joint reference and prompt pool.

The two blocks are embedded under the ten training encoders of Appendix B, each image feature concatenated with its caption’s frozen SigLIP2 text embedding [Tschannen et al., 2025] and compared under one Gaussian kernel at of the median-heuristic bandwidth with the text component weighted at ; the stack is compressed once into an -landmark Nyström reference per encoder. The generator’s conditioning pool mirrors the reference: the COCO captions together with the GenEval prompts replicated so the GenEval share of the pool matches that of the reference, about . Aligning the generated and reference prompt distributions keeps the match well-posed, its optimum reached when the two coincide.

E.2 DMD2 baseline

The DMD2 [Yin et al., 2024a] baseline distills the same four-step FLUX.2 [klein] teacher into a one-step student. The released DMD2 targets an SD-UNet under -prediction; we re-implement it for klein’s flow-matching parameterization with the method intact: three networks, the one-step generator under training, a trainable critic estimating the student distribution’s score, and the frozen four-step teacher, the DMD gradient being the teacher-minus-critic score difference, the critic updated every step and the generator every fifth. The student is initialized by regressing the teacher to a single step along its sampling ODE, after which distillation proceeds. We train at a global batch of at on four H200 GPUs with AdamW and no EMA.

Timestep schedule.

The one change klein’s parameterization forces is the distillation timestep distribution. klein is guidance-distilled and its velocity is faithful only at high noise, the four native sampling nodes; drawing the timestep uniformly reaches a low-noise regime where the teacher collapses to the mode-averaged posterior mean and drives the generator below its initialization. Mapping the uniform draw through klein’s signal-to-noise shift with the empirical that matches the native node spacing, with a learning rate of and a short warmup, turns this divergence into a student that exceeds the teacher on GenEval.

Configuration and result.

Distillation quality is set mainly by the prompt distribution. Training on a broader LAION caption pool rather than COCO captions alone lifts GenEval at every checkpoint and softens the peak-then-erode profile of distribution-matching distillation, a -point collapse becoming a -point plateau. The reported baseline is the best configuration, the LAION-prompt run at its -step peak, GenEval and PickScore (Table 8); a variant adding a GAN term on real latents was metric-neutral, its discriminator separating real from generated latents so fast that the adversarial gradient vanished against the distribution-matching one, and is not reported. The student peaks in about H200 GPU-hours.

Appendix F One-step text-to-image samples

Figure 7 shows additional single-step iRDM generations from the post-trained four-step FLUX.2 [klein], each a image produced in one network evaluation.

Appendix G Qualitative comparison

Figure 8 places uncurated iRDM samples beside pMF-H FD-SIM [Yang et al., 2026], the strongest external one-step baseline by , across five ImageNet-256 classes spanning a bird, an animal coat, a deformable garment, a rigid man-made object, and a natural landscape. Within each method the enlarged image is one sample and the adjacent grid holds ten further draws under the same class label, taken as the first released draws without cherry-picking. Per sample the two models are hard to separate by eye, both reaching sharp, on-class images; this is precisely why a distributional metric is needed, as the separation of against in Table 1 is not visible in any single row.