Skip to main content
Back to timeline
arXivSource publication:

Ensembles stop trusting one round of voting: Re-Present re-asks only disputed questions, lifting the 7–9B ensemble from 80.31% to 81.51%

Synopsis

In heterogeneous LLM ensembles, a correct member answer often exists but single-round aggregation fails to recover it. Re-Present uses single-round log-linear pooling as its backbone and re-presents only disputed multiple-choice questions under two alternative candidate-space views, a top-two view keeping the two highest-scoring candidates and a reversed view flipping option order, then merges the newly elicited candidate-level probabilities with the original evidence. Across seven multiple-choice benchmarks and two five-model ensembles, average accuracy rises from 80.31% to 81.51% (7–9B) and from 86.67% to 87.46% (14–32B), with lower extra token overhead than self-consistency under the same trigger.

Source-provided article image: RE-PRESENT: Evidence Triangulation across Alternative Candidate-Space Views for Efficient LLM Ensemble Inference
arXiv

Interpretation

Ensemble inference gains a route that needs no trained router: ask again under a different candidate presentation for questions where members disagree, then fold the new evidence into the final prediction. On seven multiple-choice benchmarks, parameter-free log-linear pooling has the highest average accuracy among the evaluated single-round routing and ranking methods, reaching 80.31% for the 7–9B ensemble and 86.67% for the 14–32B ensemble. On the same benchmarks, an oracle that counts a question correct whenever any member answers correctly reaches 92.93% and 95.40%, so a correct answer is often present but single-round aggregation does not extract it.

The procedure has two stages: five models answer independently and their candidate probabilities are combined by log-linear pooling, and the second stage runs only when the largest agreement group has at most three members. For each disputed question the second stage builds two views independently, a top-two view keeping the two highest-scoring candidates in their original relative order and a reversed view keeping all candidates but reversing presentation order and relabeling them. Each model answers both views without seeing other models' responses or its own earlier answer; the new candidate-level distributions are mapped back to original candidate identities and merged with the original prediction using a weight selected on a disjoint training split, with both ensembles selecting the same weight.

The two views supply complementary rather than redundant evidence, and combining them beats either view alone. With only the reversed view, average accuracy is 81.19% and 87.09% for the 7–9B and 14–32B ensembles; with only the top-two view it is 80.91% and 87.16%. Using both views together reaches 81.51% and 87.46%, and the two views' error indicators are only moderately correlated.

Perspective

The demonstrated scope is seven multiple-choice question-answering benchmarks (MedQA, MedMCQA, MMLU, ARC-C, OpenBookQA, MathQA, HeadQA) and two heterogeneous ensembles of five models each, with 1,000 test questions per benchmark; the method relies on candidate-level probabilities and reorderable options, and the authors list extension to open-ended tasks as future work.

Prediction stability is associated with correctness but does not guarantee it, and the authors caution that stable predictions should not guide high-stakes decisions without expert oversight, particularly in medical settings; the disagreement trigger is a heuristic, and the analysis supports it without establishing its optimality relative to confidence-based or learned triggering policies.