DreamRealm turns a model's own hallucinations into edited images: MMVP pair accuracy rises 11.34 points on Qwen3.5-9B
Synopsis
Multimodal large language models still answer against the image when correct responses hinge on subtle visual details, and existing contrastive supervision is usually built from retrieved or generated images rather than the model's own confusions. DreamRealm mines the incorrect responses a target model assigns high likelihood, minimally edits the original image so each response becomes valid, and trains the model to assign every response to the image that supports it. On Qwen3.5-9B the method improves all seven metrics across six visual perception and hallucination benchmarks, including 7.26 percentage points on HallusionBench accuracy and 11.34 points on MMVP pair accuracy.
Interpretation
The training signal moves beyond correcting wrong responses under a fixed image, teaching the model to map a hallucinated response to the one image that supports it. Earlier post-training methods treat the image as fixed and suppress hallucinations through response correction or preference learning; another line builds visual contrasts by retrieval, generation or editing, but those contrasts are not built around the target model's own hallucinations. On Qwen3.5-9B, DreamRealm improves all seven metrics across six visual perception and hallucination benchmarks, including 7.26 percentage points on HallusionBench accuracy and 11.34 points on MMVP pair accuracy.
The pipeline first makes the target model expose its own errors, then builds an image that makes each wrong response valid, and finally trains on cross-paired images and responses. RealmForgePipe expands candidate wrong responses with high-temperature sampling and enumeration of key attributes, ranks them by likelihood, and has a verifier model propose three editing instructions of increasing extent that an image generator executes, with blind checks confirming the edited image supports the response. RealmSet is built from Qwen3.5-9B hallucinations, spans 22 perceptual question categories and holds 28,960 human-approved edited examples, with 27,038 for training and 1,922 held out, split by original-image family.
The objective arranges original and edited images with candidate responses into a likelihood matrix and contrasts along both the response and image directions. RealmMatch applies supervised learning to matched image-response pairs and adds row-wise and column-wise contrastive losses: the row term favors the matching response within a family, while the column term, with response wording held fixed, favors the supporting image over other images in the family. With full RealmForgePipe data fixed, RealmMatch improves all seven metrics over the raw model and is best on six; SFT alone is best on AMBER, DPO is strong on MMVP but lowers HallusionBench, and GRPO is more balanced but still behind RealmMatch.
Perspective
The demonstrated scope is Qwen3.5-9B with RealmSet built from its hallucinations, plus cross-model training of Qwen2.5-VL-7B, Gemma-4-E4B-it and InternVL3.5-8B on the same data; evaluation covers six public benchmarks, HallusionBench, MMVP, AMBER, MMHal-Bench, CV-Bench-2D and ObjectHal, plus a held-out 5,000-example in-distribution RealmSet test split.
Candidate likelihood scores are relative weights within the candidate set, not calibrated absolute hallucination probabilities, and depend on the pool's coverage and granularity; reusing one RealmSet across models yields smaller gains than the matched setting, showing hallucination spaces overlap but are not fully interchangeable; construction depends on the image-editing model's capability, some scenes remain hard to edit, and image generation and human review add cost.
