Skip to main content
Search

# dataset generation

3 related articles

Subscribe to updates
  1. arXiv

    Agent training data no longer needs pure expert demos: JudgeMix judges each step and lifts pass@1 by up to 8 points on AppWorld

    Supervised fine-tuning of AI agents on expert trajectories leaves training data mismatched with the target model's inference-time state distribution, so the model is brittle once it deviates from the ideal path. JudgeMix uses a judge to decide at each step whether the target model's proposed action is semantically correct, hands control to the expert only when the model struggles, and reweights expert supervision by that judgment. On the AppWorld and AutomationBench tool-use benchmarks, JudgeMix reaches the highest pass@1 for both Qwen3.5-2B and Qwen3.5-9B, up to 8 and 10 points over pure expert demonstrations.

  2. arXiv

    Invent-A-Dataset generates post-training datasets from zero data, reporting 17% relative quality gains and 19% relative diversity gains

    This technical report introduces Invent-A-Dataset, a prompt-based system that goes from a dataset description to realistic and large-scale post-training datasets; evaluated in a zero-data regime against five frontier model APIs (Anthropic, Google, OpenAI, DeepSeek, Zai) across eight task types and dataset sizes up to 20K samples, it reports the highest quality (17% relative gains) and the most diverse samples (19% relative gains), with its diversity advantage widening with scale (from parity at 200 samples to 37% relative gains at 20K samples), translating into downstream training gains and higher rankings for its fine-tune across post-trained model architectures.

  3. arXiv

    FoMo turns the forking moment of a diffusion trajectory into automatic perceptual-distance labels, reaching 0.733 SROCC on PIPAL and beating metrics trained on human-annotated data

    The work proposes FoMo: forward-noising a reference image to a sampled timestep and then denoising it independently, using that forking moment as a pointwise perceptual-distance label, so training data can be generated with no human annotation, and trains reference-based IQA metrics with a RankNet-style global ranking objective; across four benchmarks and seven backbones it achieves the best average performance, with LPIPS-Alex reaching 0.733 SROCC on PIPAL versus 0.577 for the same architecture trained on human-annotated KADID-10k labels.

You’re all caught up