LLMs Can Be Confidently Wrong: CoCHI masks a few attention heads at inference and lowers calibration error on five multiple-choice benchmarks
Synopsis
Large language models often attach high confidence to incorrect predictions, while temperature scaling and similar methods adjust confidence only at the output. CoCHI sorts predictions into four classes by correctness and confidence, uses gating optimization to find attention heads causally tied to high-confidence errors, and masks only a few of them at inference without changing parameters. On LLaMA-3.1-8B, Gemma-2-9B and Mistral-7B, calibration metrics improve across five four-choice QA benchmarks while accuracy stays competitive.
Interpretation
Calibration is no longer treated only as an output-level problem; masking a few internal attention heads reduces the cases where a model is wrong yet highly confident. Temperature scaling and its variants, learned calibrators and inference-time interventions adjust outputs or representations without touching the internal mechanisms that produce overconfidence. In a zero-shot setting across three open models, LLaMA-3.1-8B, Gemma-2-9B and Mistral-7B, and five four-choice QA benchmarks, CoCHI lowered expected calibration error, negative log-likelihood and Brier score while keeping competitive accuracy.
The procedure first splits predictions into four classes by correctness and confidence, then finds the attention heads that causally shape each class's behavior. Causal head gating was previously used to explain the functional roles of attention heads; here it is applied per calibration class, comparing gate values under sparse and retention regularization. Facilitating heads from the high-confidence-wrong class and non-facilitating heads from the high-confidence-correct class are merged, ranked by attribution strength, and only the top few form the mask; the mask is applied uniformly at inference without ground-truth labels or per-sample classification.
Perspective
The demonstrated scope is four-choice question answering: confidence is defined over a fixed answer space, and experiments cover three open models, LLaMA-3.1-8B, Gemma-2-9B and Mistral-7B, on five benchmarks, MMLU, MedMCQA, OpenBookQA, MedQA-USMLE and SciQ, all zero-shot. Calibration needs a GPU, the artifact is 40 bytes, inference adds no overhead, and masked heads can be pruned permanently. The source reports calibration time inconsistently: the efficiency table gives 69.9 seconds, while the appendix gives about 39 seconds for Mistral-7B, 58 seconds for LLaMA-3.1-8B and 88 seconds for Gemma-2-9B, so no single per-model time is stated here.
In open-ended generation, long-form reasoning and multi-turn interaction, correctness and confidence are harder to define, and whether the method extends there remains to be studied; the intervention acts only on attention heads and may miss miscalibration from MLP layers, residual streams or cross-layer interactions. The threshold sensitivity analysis shows very high thresholds degrade calibration, while a moderate threshold gives the best trade-off.
