Diffusion Transformers at 4K stop losing semantics: text attention stays global while image attention returns to training scale
Synopsis
Diffusion Transformer (DiT) text-to-image models degrade semantically beyond their training resolution, and generated content loses recognizable meaning. A training-free framework keeps attention for text tokens global while confining the receptive field of image tokens to the training-resolution scale, then corrects high-frequency detail along the denoising trajectory with cross-scale and cross-temporal guidance. On Flux.1.0-dev, the method beats the listed training-free approaches on FID, KID and their patch-level versions FIDp and KIDp at 2048×2048, 2048×4096 and 4096×4096, and runs 4096×4096 inference in 14.7 time units, whereas ScaleDiff runs out of memory at that resolution.
Interpretation
High-resolution text-to-image generation no longer relies only on low-resolution structural priors as a fallback; it first repairs the model's own ability to keep semantics at high resolution. Earlier training-free DiT methods mainly relied on low-resolution structural guidance, injecting structural priors during inference rather than directly addressing the model's intrinsic semantic degradation. On Flux.1.0-dev, the method gives the best FID, KID, FIDp and KIDp among the listed methods at 2048×2048, 2048×4096 and 4096×4096; at 4096×4096 its FID is 39.687 against 43.832 for HiFlow.
After semantics are restored, the denoising trajectory still needs correction, or the global layout shows local repetition and blurred detail. With only the initial structural prior injected, the structural constraint appears only at the start, so the reverse process drifts and often produces repetitive structures, and a trajectory starting from a blurred image lacks high-frequency components. At 2048×2048, adding components one by one steadily improves global and local metrics, and the full method reaches FID 39.175, KID 0.001, FIDp 10.362, KIDp 0.002 and CLIP Score 34.01 at that resolution.
Attention scales for text and image tokens are handled separately: image tokens see only a training-scale neighborhood, while text tokens still see all tokens. In standard joint self-attention, raising the resolution grows each query token's receptive field quadratically, making attention overly uniform and focused on many irrelevant regions. Compared with overlapping patch-wise attention, the scheme needs no patch overlap or smoothing fusion and yields clearer details such as background trees in the comparison; the speedup mainly comes from replacing global attention with local attention.
Perspective
The demonstrated scope is training-free high-resolution generation with Flux.1.0-dev at 2048×2048, 2048×4096 and 4096×4096, evaluated with 1K high-quality captions and FID and KID computed against 10K real images from LAION-High-Resolution. The method is also applied directly to the Aidma and BWK customized model variants and improves their high-resolution output.
On the image-text alignment CLIP Score, the method is marginally lower than ScaleDiff at the 2048×4096 setting, a competitive level rather than a uniform lead. FID and KID require resizing images to a fixed size and may not capture high-resolution detail, so patch-level FIDp and KIDp are also computed.
