Skip to main content
Back to timeline
arXivSource publication:

Agent teams need not grow into one big group: sampling ten-agent squads in parallel beat independent agents on 10 of 12 tasks at a 100-agent budget

Synopsis

Duke University, Modal Labs, Princeton University and New York University spend test-time compute on parallel agent teams: instead of sampling only independent agents or pouring the whole budget into one large team, the method samples many ten-agent teams in parallel and keeps the best team's result. On 12 long-horizon tasks with a 27B model and a budget of 100 agents, ten-agent teams beat 100 independent agents on 10 of 12 tasks and beat a single 100-agent team on all four tasks where such large teams were run, by 8% to 30%.

Source-provided article image: Test-Time Compute Scales Better with Parallel Agent Teams
arXiv

Interpretation

Test-time compute gains a middle setting: split the budget into several independent small teams, let each team collaborate through a shared forum, and keep only the highest-scoring team. The two common routes each occupy one end: best-of-n samples many independent agents and keeps the best result but saturates quickly, while one large collaborating team scores higher yet stops improving as it grows. Across 12 long-horizon tasks, at a budget of 50 agents the best of five ten-agent teams beat the best of 50 independent agents on 11 of 12 tasks, and at 100 agents the best of ten teams beat the best of 100 agents on 10 of 12; on the four tasks with a single large team, ten-agent teams won all four at a 100-agent budget by 8% to 30%.

Small teams work because collaboration raises the mean score of one sample while parallel sampling turns the variance across samples into gains. Independent agents draw many samples with a low mean, while one large team has a high mean but is a single sample that collects no gain from variance, and its mean fell on all four tasks from 64 to 100 agents. A simple best-of-n statistical model writes the expected best score as a mean plus a variance gain; on Graph Node Classification a ten-agent team averages 79.0 against 52.1 for one agent, and the model predicts the sign of the teams' gain on 9 of 12 tasks.

Team members exchange progress spontaneously through a shared forum, with no assigned roles and no manager. Replacing the forum with an orchestrator that assigns work and collects reports made orchestrated teams score below both flat teams and independent agents at every budget on three tasks. On Borden Source Inversion the best of ten orchestrated teams scored 68.4, below the average flat team's 76.5; in flat teams agents agreed with a named teammate more than three times as often as they disagreed.

Perspective

The findings come from 12 long-horizon optimization tasks with automatic scorers, spanning nine domains from GPU kernels to formal proofs and exploit development, with each agent running for 12 hours in the Codex harness and Qwen3.8-27B as the main model; single teams of 16 to 100 agents ran on only four of the tasks, with one to five runs per size.

The statistical model is only qualitative where many attempts fail or scores hit a ceiling; team pools hold only 10 to 11 runs per task; the forum index for 64- and 100-agent teams listed only the newest posts and unread ones; the study uses one agent harness and varies session length on one task.