Preference-optimized vision-language models say less and see less: early stopping of the base model reproduces 58% to 86% of their hallucination gain
Synopsis
Four public preference-optimized checkpoints of LLaVA-1.5-7B lower object hallucination but also drop object recall on CHAIR-500 from 80.5 to between 65.1 and 76.2, with no significant set-level object F1 gain. Letting the base model stop earlier through a positive end-of-sequence logit bias until it matches each checkpoint's recall reproduces 58 to 86 percent of the CHAIRi gain of RLAIF-V, CHAIR-DPO and CSR on two disjoint 500-image COCO sets. Making the checkpoints write longer recovers only 2 to 9 percent of the lost recall, and 94 to 96 percent of the added object mentions repeat categories already named.
Interpretation
The preference-optimized checkpoints name fewer object categories while lowering the hallucination rate: all four drop CHAIR-500 recall from the base model's 80.5 to between 65.1 and 76.2 and name 2.1 to 2.8 distinct object categories per caption against 3.2 for the base model, while captions stay 75 to 93 words long. A hallucination rate counts wrong objects among mentioned objects, so mentioning fewer objects lowers it without better image understanding; the distinct-object hallucination rate is 4.9 to 19.1 points lower for every checkpoint with all eight intervals excluding zero, while the set-level object F1 in the same unit has intervals that all include zero, and CSR loses set-level F1 on the held-out set with an interval below zero. CHAIR-500 uses 500 random COCO val2014 images and AMBER its 1,004 generative images, with a separate 500-image COCO held-out set; all models run in the HuggingFace LLaVA implementation, where the base model, CHAIR-DPO, CSR and HA-DPO give identical greedy captions to the original code on 14 to 16 of 16 images.
Adding a positive bias to the base model's end-of-sequence logit so it stops earlier until its recall matches a preference-optimized model reproduces most of that model's CHAIRi gain, so much of the gain does not require preference training. At matched recall the stopping share is 61, 77 and 86 percent on CHAIR-500 and 20, 64 and 103 percent on AMBER for RLAIF-V, CHAIR-DPO and CSR, with all six intervals excluding zero; matching caption word count with the same bias reproduces only 0 to 24 percent, because at equal recall the preference-optimized captions are 1.3 to 1.8 times longer than those on the stopping curve. On two disjoint 500-image COCO sets with all models decoded at batch size 16, the shares are 58, 61 and 74 percent and the stopping-controlled gains are 4.99, 4.61 and 2.46 CHAIRi points with intervals excluding zero; HA-DPO has a raw gain of only 2.32 points and its stopping-controlled interval includes zero.
Object categories lost by the preference-optimized captions do not return when the model writes longer: with a negative end-of-sequence bias and up to 512 tokens, RLAIF-V, CHAIR-DPO and CSR recover only 2, 9 and 4 percent of the lost recall, and only 6, 4 and 4 percent of added object mentions name a new category. On the CHAIR development set the base caption of the same image still names 47, 48 and 47 percent of the ground-truth categories these three models miss, while their de-biased captions cover only 2, 13 and 2 percent, so the lost categories are absent from the preference-optimized caption rather than cut off by the stopping decision. The probe uses greedy decoding, a 512-token limit and a fixed caption prompt; no bias brings recall within one point of the base recall in any of ten calibration folds for the three models, and fewer than 5 percent of captions reach the length limit; HA-DPO loses only 4.5 recall points and recovers 58 percent, but 60.6 percent of its captions reach the limit at the matched bias.
A training-free base-guided object-category restoration appends base-only categories that the base model names with high confidence to the preference-optimized caption in a fixed sentence, bringing three checkpoints within one point of base recall on the held-out set while their distinct-object hallucination rate stays below the base model. After restoration RLAIF-V, CHAIR-DPO and CSR reach recall of 79.2, 79.5 and 79.7 against 80.1 for the base model, and their distinct-object hallucination rates of 21.6, 25.0 and 25.8 are 5.24, 1.85 and 1.08 points below the base model's 26.9 with all three intervals excluding zero, though above the original preference-optimized captions, so only part of the preference-optimization gain is kept. Thresholds are chosen on the CHAIR development set and fixed before the held-out set is generated; against random appending of base-only categories, restoration has a distinct-object hallucination rate lower by 3.77, 1.59 and 1.59 points; only 44.8, 41.4 and 44.6 percent of appended categories are in the ground truth, and base confidence separates correct from hallucinated candidates with an AUC of 0.68 to 0.69 that falls to 0.52 to 0.54 when the image is swapped.
Perspective
The findings rest on one backbone, LLaVA-1.5-7B, greedy decoding and one detailed-caption prompt, and the two COCO image sets are disjoint but from the same dataset; the stopping curve belongs only to the end-of-sequence-bias policy family, and caption length and object density still differ at equal recall; restoration covers only the object categories or noun lemmas each benchmark recognizes, and the appended sentence is a template, so only category restoration is evaluated, not caption fluency.
The stopping share is a reproduction ratio and makes no causal claim about the training effect; all intervals are nominal 95 percent intervals without correction for multiple comparisons; attribute and relation hallucination and checkpoints that failed validation are outside the study; more than half of the appended categories are hallucinated, and the advantage over the base stopping curve holds only for RLAIF-V near base recall and for RLAIF-V and CHAIR-DPO at an intermediate recall, with no such advantage found for CSR.
