Emotion recognition skips step-by-step reasoning: UEmo-D1 decides once and outputs option probabilities, averaging 75.6% across six tasks
Synopsis
Multimodal emotion recognition has split into two routes: task-specific models infer quickly but degrade when the task changes, while multimodal large language models generalize broadly at the cost of explicit reasoning and computation. UEmo-D1 maps inputs and candidate options directly to an option probability distribution at a single decision position, with no intermediate reasoning; on UEmo-Data, covering six task types and 41 evaluation subsets, it reaches 75.6% average accuracy, 13.3 points above runner-up Gemini-3.5-Flash, with an average ECE of 4.7.
Interpretation
Multimodal emotion recognition can now skip step-by-step reasoning and give a single probability judgment over candidate options, with one model handling facial expressions, speech, image-text posts and conversational video alike. The two prior routes each had a weakness: task-specific models are cheap to run but degrade noticeably when input modalities are disrupted, the label space changes, or the input domain shifts; multimodal-large-language-model approaches generalize more broadly but rely on explicit reasoning and pay a higher computational cost. Across the 41 evaluation subsets of UEmo-Data, UEmo-D1 averages 75.6% accuracy, 13.3 points above runner-up Gemini-3.5-Flash at 62.3%; across the six tasks its accuracy gains over the strongest baseline in each task range from 4.8% to 18.4%.
After reading the instruction and candidate options, the model emits a single decision token, and the probability distribution at that position serves as the confidence for each option, with no explanatory text generated. The recipe initializes from Qwen2.5-Omni-3B and replaces the causal attention used in pretraining with bidirectional attention, so every input token attends to all others at every layer and cross-modal context modeling is strengthened. Training runs for one epoch and takes about 20 hours on four NVIDIA RTX 4090 GPUs, with LoRA rank 16 and batch size 64; probability normalization is confined to the valid option set, and temperature scaling is fitted on a held-out validation set without updating model parameters.
Confidence calibration is where this recipe separates itself from reasoning-style models, since its option distributions sit closer to actual accuracy than most compared models. Compared models derive option probabilities from first-token logits in decision mode; UEmo-D1's average ECE is 4.7, about one-third of the next-best result, and its Brier score is 33.5, only 60% of the next-best score. When option order is cyclically shifted while instructions and multimodal content stay unchanged, UEmo-D1's average accuracy moves from 75.6% to 75.6%, whereas AffectGPT, Nano-EmoX and OneEmo drop by 2.0 to 5.4 points on average.
Perspective
Evaluation covers the test split of UEmo-Data: 41 subsets and 19,072 samples, with inputs in image-text, video-text, audio-text and video-audio-text configurations and candidate class counts from 2 to 25; the conclusions are limited to this data collection and the decision-mode setting.
Under the modality-conflict conditions of CA-MER, UEmo-D1 reaches 54.0% accuracy on audio-aligned samples but falls to 41.2% on video-aligned samples, indicating a remaining bias toward acoustic evidence when modalities disagree; it does not lead on visually distinctive subsets such as Abstract, ArtPhoto and UnbiasedEmo, nor on the MUStARD subset; calibration deviations are larger in low-confidence bins.
