Skip to main content
Search

# GRPO

120 related articles

Subscribe to updates
  1. NVIDIA releases Nemotron for Science with a post-training recipe: on drug-metabolism prediction, SFT lifts accuracy 12% and RL another 17%

    NVIDIA released Nemotron for Science, a collection of models, datasets, tools and recipes that lets research teams turn experimental records into a verifiable-reward environment and adapt a general multimodal model into a domain-specific science agent. In a drug-metabolism example, the model takes a parent compound and its transforming enzyme and outputs the metabolite SMILES; at a 16,384-token output budget, supervised fine-tuning raised Nemotron for Science accuracy by 12% and RL post-training lifted it a further 17%. The recipe also offers an optional teacher-model distillation warm start for cases where RL converges slowly.

  2. NVIDIA's Vera Rubin debuts on MLPerf Training with the fastest time on six of seven benchmarks, about 2x per GPU on DeepSeek-V3

    NVIDIA's Vera Rubin architecture entered MLPerf Training v6.1 for the first time, and its NVL72 platform posted the fastest training time on six of the seven benchmarks, up to 2x the highly optimized GB300 NVL72 baseline. On the 671-billion-parameter DeepSeek-V3 workload, Vera Rubin NVL72 cut training from 8.03 to 4.14 minutes at 1,024 GPUs, while the round's fastest absolute result, 1.38 minutes, came from a 10,240-GPU GB300 NVL72. NVIDIA reports above 97% overall scaling efficiency for Vera Rubin from 256 to 1,024 GPUs.

  3. arXiv

    EmoReward stops emotional video RL from trusting fixed rewards: last run's traces set the next run's weights

    Emotional video generation has no single correct target, and fixed fusion of visual, motion and text-alignment rewards captures emotional intent poorly. EmoReward distills reward traces from one completed training run into per-emotion-regime reliability estimates, converts them into bounded weight multipliers, and combines them with current-video instance gates in the next run without updating any reward-model parameters. On HunyuanVideo 1.5 with DanceGRPO, FlowGRPO and SAGE-GRPO, VideoAlign Overall rises under all three algorithms, including SAGE-GRPO with EmoReward moving from -0.8269 to -0.2624.

  4. arXiv

    A decision model that acts badly can still teach: JEV scores whole trajectories and lifts a 1.5B policy from 72.8% to 82.9%

    Decision models are usually judged by how well they act, yet the same models are pulled toward an easier subgoal when acting on ALFWorld. Used instead as a teacher that sees the whole trajectory after an episode and is asked whether each action helped, JEV raises a Qwen2.5-1.5B policy trained with GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop, while JEV as an actor completes only 42.9% of ALFWorld tasks.

  5. arXiv

    Agent RL stops scoring whole trajectories: ORSD writes its own rubrics to assign credit token by token

    Online Rubric Self-Distillation (ORSD) has the current policy generate natural-language process rubrics from differences among a group of its own trajectories, lets a frozen verifier pick the bundle whose rubric satisfaction best correlates with task success, and uses that bundle as privileged context: the log-likelihood gap for the same tokens with and without the rubric becomes a token-level correction to the trajectory-level GRPO advantage. On ALFWorld and WebShop, Qwen3-1.7B and Qwen3-4B reach the best overall results among the compared methods under the same environment-interaction budget.

  6. arXiv

    Traffic lights skip the perception pipeline: Traffic-VL reads raw camera views and cuts average queues 10.4% over three weeks at ten intersections

    Traffic signal control has relied on structured traffic states from dedicated perception pipelines, or on converting simulator images into intermediate representations before deciding. Traffic-VL is an eight-billion-parameter vision-language model that maps synchronized multi-view images and natural-language phase descriptions directly to phase choices, trained with SynTra-V, which turns physics-based simulation into semantically audited photorealistic surveillance imagery. It matches the structured-state Traffic-R1 on five closed-loop benchmarks, reaches 82.1% phase-selection accuracy on real surveillance images, and over three weeks at ten intersections reduced average queue length by 10.4% relative to historical manual operation.

  7. arXiv

    Video models learn to check the evidence before reasoning: EVAThink hits 63.5% average at 32 frames with 6.7K samples and about 62% shorter output than always reasoning

    EVAThink turns whether a sampled set of frames is enough to answer directly into a training signal: for the same question and answer it builds evidence-dense and evidence-sparse views of a video and keeps only pairs where direct answering degrades on the sparse view but explicit reasoning recovers accuracy, then uses two-stage reinforcement learning to teach the model to switch between direct answering and reasoning. Trained on only 6.7K samples, it reaches 63.5% and 65.6% average across six video understanding benchmarks under fixed 32- and 64-frame budgets, both above the Qwen3-VL-8B-Instruct base; at 64 frames the adaptive mode beats direct-only and think-only by 2.3 and 3.6 percentage points and cuts output length by about 62% relative to think-only.

  8. arXiv

    Agent memory gets its own compressor: SumOne learns from masked reconstruction and cuts search-task cost to 22% of full context

    Long-horizon language-model agents accumulate ever-longer interaction histories, so context compaction is necessary, yet existing learned compaction methods often depend on downstream task outcomes or solver behavior. SumOne is a standalone 2-billion-parameter summarizer trained with a task-agnostic reward built from masked reconstruction and optimized with GRPO, using only raw trajectories and no reference summaries or downstream task execution. On DeepSearchQA it cuts cumulative agent input from 2.83M to 591K tokens with F1 0.48 versus 0.52 for full context; on SWE-bench Verified it reaches a 0.59 resolved rate versus 0.63, with input falling from 4.10M to 1.09M tokens.

  9. arXiv

    Tool calls get worse with training: TACA supervises the reasoning-to-call transition and lifts six-benchmark accuracy by 4.61 points

    In multi-turn tool-use reinforcement learning, reasoning accuracy can rise early and then fall below the non-agentic baseline; TACA traces this to negative aggregate credit on tool-call tokens and instead supervises the step where reasoning turns into a tool call. Against MT-GRPO it raises mean accuracy over six reasoning benchmarks by 4.61 and 3.61 percentage points for Qwen2.5-1.5B and 3B, while tool execution success during training stays at 86.28%.

  10. arXiv

    Chest X-ray reports that show their work: ARIVE inspects anatomy before diagnosing, matching Qwen3.5 in blinded grading

    Automated chest X-ray reports often read fluently while leaving unclear which image evidence supports which finding. ARIVE interleaves anatomical inspection with diagnostic reasoning before composing the report, trained on 159,416 multimodal chain-of-thought trajectories built from MIMIC-CXR. On MIMIC-CXR and the abnormality-only MIMIC-ABN it scored highest overall on clinical-efficacy measures among the compared models, and in blinded grading of 100 cases its report quality was close to Qwen3.5 (mean 3.47 versus 3.51, against 3.88 for reference reports).