Skip to main content
Search

Wang Ma

1 related articles

Subscribe to updates
  1. arXiv

    Black-box LLM uncertainty splits in two: a two-level ensemble separates vague prompts from missing knowledge

    Large language model uncertainty has been collapsed into a single consistency score that cannot say whether the model misread the question or lacks the answer. A two-level ensemble treats repeated stochastic decodes of one prompt as the inner level and meaning-preserving rewrites as the outer level, then uses the law of total covariance in sentence-embedding space to split aleatoric from epistemic uncertainty using only black-box sampling. Across five short QA benchmarks and four instruction-tuned models it is competitive with strong white-box baselines, and paired interventions make the two components respond separately to prompt ambiguity and knowledge gaps.

You’re all caught up