Skip to main content
Search

# Value Estimation from Actor's Internal States

1 related articles

Subscribe to updates
  1. arXiv

    POISE predicts value baselines from a model's own internal states, beating GRPO and PPO with more stable training on a six-domain verifiable-reward corpus

    The work introduces POISE (Policy Optimization with Internal State Value Estimation), which uses a lightweight probe reading signals already computed during the forward pass to predict the value baseline, and a cross-rollout construction that predicts each rollout's value from an independent rollout's internal states to preserve gradient unbiasedness; on Qwen3-4B and OLMo3-7B-Instruct-DPO across a six-domain verifiable-reward corpus, POISE outperforms other RLVR baselines with more stable training, and the probe matches a separate LLM-scale value model, generalizes to various tasks, and remains accurate as the policy scales.

You’re all caught up