Professional image quality assessment moves past a single score: NTIRE 2026 RAIM Track 1 asks multimodal models to pick the better photo and explain why
Synopsis
Track 1 of the NTIRE 2026 3rd RAIM Challenge recasts professional image quality assessment as pairwise comparison: models must pick the visually superior image from a high-quality pair shot on different devices in the same scene and give an expert-style written rationale. The track supplies 100 annotated training pairs, a 102-pair validation set and a 101-pair test set, counted separately for portraits and scenes, with test subjects absent from training. Final ranking fuses comparative accuracy with reasoning quality, and champion IH-VQA reached a final award score of 0.7305.
Interpretation
The object of professional image quality assessment shifts from a scalar score for one image to which of two high-quality images is better, and why. Conventional practice regressed complex visual characteristics into a mean opinion score, which struggles to separate subtle differences among uniformly high-quality images and cannot say why one image is superior. The track built pairs of the same scene captured with different cameras or smartphone models, and professional imaging effect designers marked the superior image as a binary multiple-choice question with written reasoning and localized crops highlighting the main subjects.
Participating systems must produce an explanation aligned with expert judgment alongside the choice, so evaluation splits into comparative accuracy and reasoning quality. Reasoning quality is computed only on samples the model got right, as a combination of BLEU-4 and ROUGE-L, so elaborate justifications for wrong choices earn nothing; a separate LLM judge scores semantic alignment. The Phase 2 score uses multiplicative fusion, accuracy times the sum of 0.7 and 0.3 times the conditioned text score, Phase 3 adds 0.5 times the LLM judge score, and the final ranking averages the two phases equally; Phase 2 leader IH-VQA recorded accuracy 0.9118 and conditioned NLG 0.2050.
Final-stage teams largely fine-tuned multimodal large language models, with GRPO reinforcement learning and voting ensembles as the common strategies. Phase 2 drew 85 active participants and 2530 submissions, and 57 teams advanced to the single-attempt Phase 3 evaluation. VCIP Pi Group ranked first on Phase 3 score at 0.7679, with LLM score 0.5535 and accuracy 0.6733; IH-VQA's Phase 3 score was 0.7667 with accuracy 0.7129.
Perspective
The track evaluates binary selection and written rationale on high-quality image pairs of the same scene shot on different devices, with 100 training, 102 validation and 101 test pairs counted separately for portraits and scenes; conclusions are limited to that data split and the single-attempt final setting.
The report notes that models generally manage the binary choice, but generating rationales aligned with the experts' photographic dimensions remains a clear difficulty; how to evaluate those rationales across diverse image content and shooting conditions is also an open problem, since suitable criteria vary with the visual context and artifacts in each pair.
