Legal judgment models keep guilty verdicts after the decisive fact changes: PROSE cuts damage to supported rulings from 17.5% to 8.5%
Synopsis
Legal judgment prediction models often keep a guilty verdict after a legally decisive condition has been altered, a tendency the authors call guilty bias. PROSE frames the correction as a paired constraint: the unsupported member of a case pair should flip, while its counterpart, differing in one decisive condition, should stay. On Qwen3-8B and GLM-4-9B it performs this selective repair through a one-time offline parameter edit, and at matched repair rates it reduces damage to supported verdicts from 17.5% to 8.5% on Qwen and from 24.3% to 7.1% on GLM.
Interpretation
The correction target shifts from lowering guilty predictions overall to selective repair between two versions of the same case: the unsupported version should flip to charge-not-established, while the supported version, differing in one decisive condition, should keep its verdict. Earlier routes, including non-guilty supervision, defense-oriented interaction, self-refinement and debate, aim at aggregate error reduction, and a lower overall error rate can come from a general downward shift in guilty scores that raises the false-non-guilty rate. On the CAIL-MC Pairwise Repair–Preservation Track, PROSE reaches higher pairwise safe repair than both the legal judgment prediction baselines and the model-editing baselines on Qwen3-8B and GLM-4-9B, with false-guilty and false-non-guilty rates no higher than 8.61 and 15.04 on Qwen and 15.04 and 10.61 on GLM.
The repair is delivered by a one-time offline parameter update, and deployment remains ordinary single-pass inference with no paired inputs and no repeated deliberation per query. Data-based correction requires multi-step training, while inference-time critique and debate repeat deliberation for every case. Offline cost is 1.1 GPU-hours for Qwen and 2.1 GPU-hours for GLM on one RTX 4080 SUPER; against a uniform margin shift calibrated to the same repair rate, PROSE reduces the fraction of base-correct supported-guilty cases flipped to wrong from 17.5% to 8.5% on Qwen and from 24.3% to 7.1% on GLM.
Evidence sensitivity survives a change of corpus and construction: with only an allegation the model tends not to establish the charge, and with full facts it still does. Full supervised fine-tuning, strongest on matched pairs, reaches only 49.64–51.00 accuracy on the evidence-sufficiency track, where false-guilty errors dominate at 94.29–97.25. Across four evidence conditions built from LeCaRD and Criminal-S, PROSE obtains the best cross-corpus macro-F1 on GLM and competitive performance on Qwen; on three rewritten evidence levels of CAIL-MC test facts, Qwen's establishment rate for allegation-only inputs falls from 93.8% unedited to 10.3%, while 86.1% of fully evidenced texts are still established.
Perspective
Evaluation covers charge-conditioned verification under Chinese criminal law with two backbones, Qwen3-8B and GLM-4-9B, rather than complete judicial decision-making; the pairwise track derives from CAIL2018 and the evidence-sufficiency track from LeCaRD and Criminal-S, the latter measuring only whether the supplied text supports the charge, not evidence credibility or a judicial standard of proof.
CAIL-MC uses LLM-assisted transformations of adjudicated facts, and independent legal review reduces but cannot eliminate generation artifacts and uncertainty in legal interpretation; its coverage-oriented construction and selection of editable cases do not represent the distribution of criminal cases in practice, and the evidence track has a small source-case sample and a fixed allegation-only template. PROSE relies on paired training examples and backbone-specific configuration choices, its finite strength search may miss feasible interventions, and parameter fitting approximates their activation effects, so preservation is empirical rather than guaranteed.
