Skip to main content
Search

Xiangyu Yue

1 related articles

Subscribe to updates
  1. arXiv

    Benchmarks no longer need hand-built pipelines: Benchmark Agent auto-generates 15 benchmarks with roughly 96–98% human acceptance

    MMLab at The Chinese University of Hong Kong and collaborators released Benchmark Agent, which turns benchmark construction from a human design, collection and annotation pipeline into an agent-run process. A Benchmark Planner decomposes a user requirement into subtasks grounded in real datasets, and a Benchmark Executor realizes samples with tools. Across 15 benchmarks spanning text, multimodal and domain reasoning, five representative ones reach human acceptance rates of about 96–98% and LLM-as-Judge user-intention alignment of 68.54–81.48, so the generated benchmarks largely preserve user intent while grounding and difficulty control remain weaker.

You’re all caught up