Generative Modeling via Drifting: One-Step AI Generation Explained
A new paper co-authored by Kaiming He achieves state-of-the-art image generation in a single forward pass — no diffusion steps, no ODE solvers. Here's the mechanism behind it, and what it means for AI-generated image detection.
The Problem: Iteration at Inference Is Expensive
Diffusion models and Flow Matching have dominated generative AI for good reason — they produce stunning results. But their fundamental architecture requires multi-step inference: you run the network 20, 50, sometimes 100 times per image. Each step is a forward pass through a large transformer. The compute cost adds up fast.
Distillation methods try to compress those steps down to fewer iterations. They work, but they're still built on the same SDE/ODE scaffolding — you're trimming a process, not replacing it.
Drifting Models take a different approach entirely: eliminate iterative inference by moving the iteration somewhere else.
The Core Insight: Move Iteration to Training
Neural network training via SGD is already iterative. Drifting Models exploit this — the evolving sequence of model checkpoints becomes the iterative process, not inference.
The goal of any generative model is learning a mapping f such that pushing a simple prior distribution through f produces the target data distribution:
In diffusion and flow matching, the network learns a vector field. At inference time, you integrate that field iteratively — Euler steps, Runge-Kutta, whatever the chosen solver — to get from noise to image.
Drifting Models discard that solver entirely. Instead, a specialized training objective forces the distribution of generated samples to "drift" toward the real data distribution as training progresses. By the time training ends, the network outputs final images directly from noise — one pass, done.
The Mathematics: Attraction, Repulsion, and Equilibrium
The drifting mechanism is controlled by a drift field V that specifies how a generated sample f(ε) should move to become more data-like. Two properties make this work.
Anti-Symmetry and Equilibrium
For training to converge rather than oscillate, the drift field must self-extinguish when the job is done. The authors enforce this via anti-symmetry: when the generated distribution matches the data distribution exactly, the drift force is zero:
This is meaningfully different from diffusion. Diffusion models follow a fixed noise schedule regardless of whether the model is already generating good data. Drifting Models are self-regulating — gradient updates slow naturally as the generated distribution approaches the real one.
The Stopgrad Training Objective
The loss function has an elegant structure:
The stopgrad operator is the key move. It computes where the current sample should drift — the target coordinate — then freezes it so gradient flow doesn't collapse the computation into a trivial solution. The optimizer then pulls the network's parameters to output that coordinate directly on the next step. Each training iteration, the network gets a little closer to generating the right answer in one shot.
Designing the Drift Field: Avoiding Mode Collapse
The drift field is constructed from two competing forces:
Attraction (V⁺)
Real data samples pull generated samples toward them. This is the signal that pushes outputs toward the data manifold.
Repulsion (V⁻)
Generated samples push each other apart. Without this, all outputs would collapse to a single average-looking blob — the classic mode collapse failure.
One practical detail worth noting: computing this in raw pixel space is ineffective. The drift field is instead calculated in a semantic feature space using pre-trained encoders like MAE (Masked Autoencoders). This gives the attraction and repulsion forces access to meaningful structure rather than raw pixel values — the difference between "these two images look conceptually similar" and "these two images share similar pixel values."
The researchers frame this through a physics lens: training becomes a particle simulation, where generated samples are particles being attracted toward real data and repelled from each other until equilibrium is reached. — "Generative Modeling via Drifting," He et al.
Benchmark Results
The proof is in the numbers. Tested on ImageNet 256×256 — the standard benchmark for class-conditional image generation — Drifting Models set a new record for single-step generation:
| Model | Type | Architecture | FID ↓ | Steps (NFE) |
|---|---|---|---|---|
| Drifting Model (Latent) Best | Drifting | DiT-L/2 | 1.54 | 1 |
| Drifting Model (Pixel) | Drifting | DiT-L/2 | 1.61 | 1 |
| iMeanFlow | Flow Matching | — | 1.72 | 1 |
| StyleGAN-XL | GAN | — | 2.30 | 1 |
A 1.54 FID score with a single network evaluation beats all prior single-step methods. For context, multi-step diffusion models typically require 50–250 steps to reach comparable quality — and still often score above 2.0.
The paper also applies the drifting framework to robotics under the name Drifting Policy. There, the goal is generating action sequences rather than images. The results match 100-step diffusion policies in task success rate while running fast enough for real-time robotic control — a case where iterative inference was previously a hard blocker.
What This Means for AI Detection
Faster, cleaner generation architectures directly affect detection. The artifacts that detection systems rely on are often byproducts of how an image was generated — the specific noise patterns introduced by a diffusion schedule, the blending artifacts from flow integration steps, the frequency signatures left by iterative refinement.
Drifting Models don't have those steps. A single-pass generator trained with this paradigm produces images whose artifact signatures are fundamentally different from what diffusion or GAN-based detectors were trained to recognize.
At UncovAI, we track shifts like this closely. Our detection systems are built to adapt as generation methods evolve — not locked to the artifacts of any single paradigm. Whether the image came from 100 diffusion steps or a single drifting pass, the underlying question is the same: does this image contain signatures of a generative process? The answer doesn't change. Our approach to finding it does.
As self-supervised encoders improve — better vision transformers, stronger MAE models — they'll act as more precise compasses for the drift field. Generation quality will keep climbing. Detection needs to stay ahead of it. — Aditya, Data Scientist at UncovAI
Frequently Asked Questions
What are Drifting Models?
A generative AI paradigm that achieves single-step image generation by moving the iterative matching process from inference to training. The result is a network that outputs a final image directly from noise — no diffusion schedule, no ODE solver required.
How do Drifting Models compare to diffusion models?
Diffusion models require 20–250 forward passes at inference time. Drifting Models require one. On the ImageNet 256×256 benchmark, the latent Drifting Model scores 1.54 FID — better than all prior single-step methods and competitive with the best multi-step diffusion results.
What is the stopgrad trick and why does it matter?
Stopgrad freezes the target coordinate during the loss computation so gradient flow doesn't collapse the training objective into a trivial solution. It's what allows the optimizer to treat the drift destination as a stable target rather than a moving one — critical for stable convergence.
Why is semantic feature space used instead of pixel space?
Computing attraction and repulsion in raw pixels captures low-level color and texture similarity, not semantic meaning. Pre-trained encoders like MAE give the drift field access to higher-level structure, making the forces more meaningful and the training more effective.
Does this affect how AI-generated images are detected?
Yes. Detection systems trained on diffusion or GAN artifacts may not generalize to images from single-step drifting generators. The generation process leaves a different fingerprint. Systems that model the broader signatures of AI generation — rather than artifacts specific to one method — are better positioned to handle new paradigms like this.
A Post-Diffusion World?
Drifting Models don't just optimize an existing process — they replace it. The SDE/ODE scaffolding that has defined generative AI for the last four years is no longer the only path to high-quality generation. That's a meaningful shift, and it's worth paying attention to.
The framework is modular. Better encoders mean better drift compasses, which means better generation quality. The ceiling isn't in sight yet.
Stay Ahead with UncovAI →