Playing log(N)-Questions over Wikipedia Abstracts: Communication Efficiency Between Paired Frontier Models
This technical report ports the two-agent log(N)-Questions game of Potash and Suleman (2019) to six frontier language models, where a questioner sees N Wikipedia lead paragraphs and must identify a hidden target using exactly log2(N) yes/no questions while an answerer sees only the target and the question and replies with one word; across 408 games over document sets of 4 to 1024 paragraphs at a total API cost of $363, it finds the leading five models only marginally separable, fits their declining win rate with a single per-round reliability parameter p=0.928 in win=plog2 N, and attributes losses in roughly equal measure to answer errors and discrimination failures, with failure concentrated in communication rather than inference.