Diffusion models stop retraining per distribution: a handful of samples now steers reverse sampling to unseen tasks
Diffusion models previously learned either one fixed target distribution or a family indexed by conditioning variables fixed at training time, so an unseen distribution required retraining or fine-tuning. Treating each target distribution as a task drawn from a meta-distribution, a shared score operator now takes a small context set at inference and outputs that task's score, enabling reverse-diffusion sampling without parameter updates. The task-dependent score admits a Gaussian-kernel conditional-mean representation whose empirical Nadaraya–Watson estimate is represented exactly by the constructed transformer class; on Navier–Stokes, ERA5, QuickDraw and Food-101, low-intrinsic-dimension tasks do well while performance degrades as task complexity grows.