EEG-to-image reconstruction stops leaning on alignment alone: an image teacher supervises denoising, and the method beats its same-backbone baseline on all seven THINGS-EEG metrics
Synopsis
EEG-to-image reconstruction has long aligned brain signals to pretrained visual embeddings and used them to condition a generator. Retrieval experiments here show that better alignment does not mean better reconstruction: retrieval improved by up to 77, while high-level reconstruction gains stayed modest and low-level fidelity showed no consistent improvement. An image teacher conditioned on DINOv2 features is trained first to express fixed encoder features as intermediate denoising representations, and an EEG-conditioned student generator aligns to these dynamic targets during denoising, with the teacher discarded at inference. On ten THINGS-EEG subjects, the method beats a SiT-XL/2 baseline trained on the same Brain-HiVE embeddings without representation alignment on all seven metrics.
Interpretation
The route to better EEG-to-image reconstruction shifts from strengthening the input condition to directly supervising the generative process. Most prior methods aligned EEG to visual representations and used the resulting embeddings to condition an image generator, for example SDXL with IP-Adapter or a visual autoregressive model fine-tuned on EEG representations. Conditioning a SiT generator on representations from recent EEG-to-image alignment methods raised retrieval performance by up to 77 relative to the baseline condition, while gains in the evaluated high-level reconstruction metrics stayed modest and low-level fidelity showed no consistent improvement.
The supervision signal now comes from an image-conditioned teacher that expresses visual features as intermediate representations the generator can follow during denoising. Aligning directly to pretrained vision-encoder features, the representation alignment approach REPA, left EEG-conditioned intermediate representations poorly aligned with DINO features. Teacher and student share a SiT-XL/2 architecture; the teacher receives frozen DINOv2 features projected into every SiT block, is trained once, frozen and reused across all subjects, while the student is conditioned on EEG embeddings and aligns its block 8 to the teacher's block 25 with a token-wise cosine loss, keeping only the student at inference.
Perspective
The conclusions rest on offline reconstruction evaluation with ten THINGS-EEG subjects and four THINGS-MEG subjects, using pixel correlation, structural similarity and AlexNet, Inception, CLIP and SwAV similarity; THINGS-MEG reports descriptive results only because it has four subjects, without significance testing.
Why retrieval gains fail to carry over to reconstruction remains unanswered, and which visual cues help distinguish images during retrieval versus recover their appearance during generation is left open; the generator is still fully fine-tuned for each individual subject.
