Skip to main content
Search

# Aggregation of an Ordering with its Reversal

1 related articles

Subscribe to updates
  1. arXiv

    A multiple-choice bias score splits in two: flip rate and positional skew, so two mitigations stop looking alike

    Under cyclic reorderings that show every option at every position equally often, the pooled selection-rate standard deviation factors exactly into two label-free quantities: the flip rate, how often the chosen option changes with order, and the positional skew, how unevenly the changed answers land on letters. Across 41 open-weight checkpoints and 12 benchmarks, the flip rate tracks accuracy and output uncertainty, while the skew tracks the model's option-symbol prior. Prior correction mainly lowers the skew and aggregating an ordering with its reversal mainly lowers the flip rate; both narrow the accuracy range across correct-answer letters by similar amounts, but aggregation leaves fewer model-rank reversals and settles more items on a wrong option.

You’re all caught up