Skip to main content
Back to timeline
arXivSource publication:

Game2Player has agents build game environments while they learn to play, lifting a 9B model's progress on held-out games from 14.3% to 31.2%

Synopsis

Visual agents need actions, goals and verifiable success signals to learn games, yet human recordings are costly, gameplay videos lack actions, and adapting each game by hand takes engineering. Game2Player has a Builder agent turn web games into offline, reproducible environments that expose internal state, then lets a code-writing Coding Player and a screenshot-reading VLM Player test every task and report defects, so play both validates the environment and collects data. Trained on 568 games, 3,436 tasks and 15.7K trajectories, Game2Player-9B raises progress from 14.3% to 31.2% on the held-out PlayerBench and from 30.0% to 53.1% on the external GameWorld benchmark.

Source-provided article image: Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development
arXiv

Interpretation

Gameplay experience no longer has to come from human recordings or after-the-fact annotation; agents generate it inside adapted web games, with every step paired to a task instruction and a progress signal read from game state. Earlier routes either collected human gameplay, inferred missing actions from videos and annotated goals and outcomes afterwards, or required game-specific engineering for each new game, which limited coverage. In a controlled rebuild of 20 games with 145 frozen tasks, player reports raise correctly implemented tasks from 65% to 72% and tasks solved by code policies from 34 to 46; with the same model as a generic coding agent, only 25 of 145 task environments are correct and one task is solved.

Environments and players are developed together: the Builder packages each game into a fixed-seed, pausable offline runtime, exposes a state probe and a player-visible observation, and defines each task's start, progress and success checks. The Builder's own tests use the inputs it designed and the variables it exposed, so they can miss defects that only a real player meets, such as a key with no effect or progress that never moves. Both players attempt every task and file defect reports or observation requests, and the Builder replays the recorded episode before repairing; in There Is No Game, the Coding Player reported progress stuck at 0.9 after the letter O fell, and after the repair all nine tasks are correct and solved, while without reports all nine remain wrong.

The two players supply complementary data: code policies call no model and can be rerun from new seeds, while VLM Player episodes keep the model's action notes and thinking text. Code policies act continuously in real time and must be cut into the same step format as evaluation, whereas a VLM Player waits for a model decision at every step and records far more slowly. Together they record 15.7K trajectories and 307 hours of gameplay, with code policies contributing 87.7% of training examples and VLM Players 12.3%; in one hour of wall-clock time a code policy records 7,775 harness steps against 354 for a VLM Player.

Training weights samples by progress, concentrating supervision on the actions that actually move a task forward instead of sampling whole trajectories uniformly. Most steps in a trajectory are spent waiting, wandering or recovering from mistakes, and only a few actions produce a rise in progress. At equal example counts, progress-weighted SFT raises PlayerBench progress from 20.9% to 23.4% for the 4B model and from 23.9% to 28.6% for the 9B model, and GameWorld progress from 33.5% to 34.9% and from 31.1% to 33.9%.

Perspective

The demonstrated scope is single-player web games that run offline in a browser, spanning seven engines and more than 25 genres; evaluation uses PlayerBench, 50 held-out games with 287 tasks, and the 34-game GameWorld, with the game paused for each model decision.

Accepted environments are not guaranteed correct: acceptance relies on the Builder's own evaluation, a scripted gate and player reports, without a full human audit; the audit finds 72% of tasks built by the GPT-6.1 Terra Builder and 94% of those built by the GPT-6.1 Sol Builder correctly implemented, while human review of all 287 PlayerBench tasks found no environment or scoring errors. Paused decisions also do not measure play under real-time latency, and the Builder and players are mostly proprietary models whose versions change, so rebuilding with other models may give different environments.