Skip to main content
Search

# Model distillation

3 related articles

Subscribe to updates
  1. NVIDIA releases Nemotron for Science with a post-training recipe: on drug-metabolism prediction, SFT lifts accuracy 12% and RL another 17%

    NVIDIA released Nemotron for Science, a collection of models, datasets, tools and recipes that lets research teams turn experimental records into a verifiable-reward environment and adapt a general multimodal model into a domain-specific science agent. In a drug-metabolism example, the model takes a parent compound and its transforming enzyme and outputs the metabolite SMILES; at a 16,384-token output budget, supervised fine-tuning raised Nemotron for Science accuracy by 12% and RL post-training lifted it a further 17%. The recipe also offers an optional teacher-model distillation warm start for cases where RL converges slowly.

  2. arXiv

    GenoTrace uses codon-aware watermarks so distilled genome models stay detectable, reaching 94.5% detection in a three-seed experiment

    The work introduces GenoTrace, a codon-aware extension of green-list watermarking that modulates a teacher model's generation bias with two token-level factors—codon position and organism-specific codon usage—so that the resulting synthetic sequences remain auditable after distillation into a smaller student; in a three-seed GenomeOcean-500M-to-100M experiment the joint configuration achieves a mean audit score of 17.88 and 94.5% detection at a fixed threshold, retaining 49.0% detection after key-aware token substitution (versus 0% for the available single-seed plain-watermark comparator) and 47.0% after combined mechanism-targeted nucleotide edits.

  3. arXiv

    Mid-Harness verifies candidate actions between model and harness, lifting TMAX-9B Pass@1 on TerminalBench-Lite from 50.00% to 68.03%

    The work introduces Mid-Harness, which samples and verifies candidate actions at the model-harness boundary before forwarding one for execution while keeping the generator and harness unchanged, and studies action-level test-time compute scaling: on TerminalBench-Lite a GPT-5.6 Sol verifier raises TMAX-9B Pass@1 from 50.00% to 68.03% with 8 sampled actions, wider sampling yields little benefit under weak verification, pairwise verification performs best among the evaluated self-verification mechanisms, distilling the stronger verifier's responses into TMAX-9B further improves Pass@1, and combining action scaling with trajectory scaling (Best-of-N and Sequential Refine) reaches higher success at lower estimated token cost.

You’re all caught up