Skip to main content
Back to timeline
arXivSource publication:

Frontier LLMs can write small network weights outright: replication holds, but reading weights mostly reads the input layer

Synopsis

A three-day replication tested whether frontier LLMs can read and write the weights of small neural networks directly. GPT-6 Astra's Breakout networks scored 514, 550 and 307 across three runs, in line with the originally reported 517, and did just as well on two Breakout variants that exist nowhere else; yet when two models independently decompiled Astra's Breakout network into Python, both programs chose the same action as the network in all 49,189 game states tested, indicating that writing weights amounts to compiling a program into weights while reading weights mostly recovers the input-layer layout.

Source-provided article image: Replicating and Extending Weight-Space Programming: Frontier LLMs Compile Programs into Weights and Read Input Layers
arXiv

Interpretation

The ability of frontier LLMs to write small network weights directly reproduces, and it is not recall of a known network from training data. The original report said GPT-6 Astra wrote a 20,486-weight Breakout network in a single reply with no tools and scored 517, against 25 for a same-architecture network trained with PPO for 500 million steps. In the replication Astra's Breakout networks scored 514, 550 and 307 across three runs; on two Breakout variants that exist nowhere else, with left and right swapped and with the board upside down, the networks it wrote for those variants averaged about 515, while its standard-game network scored just 0.5 and 0.0 on them.

What is new is translating programs into weights: the written networks behave like compiled programs rather than the distributed representations of trained networks. Earlier models could write a usable Python policy roughly one generation before they could write the same policy as network weights, and the late-2025 Claude Opus 4.5 mostly could not even write good programs. Claude Opus 5.5 and Astra each independently decompiled Astra's Breakout network into Python, and both programs chose the same move as the network in all 49,189 game states tested, averaging 502.9 over 1,000 games, exactly as the network does.

Reading weights is mostly reading the input layer: models recover where objects sit on the board but often miss the game mechanics. In the original experiments every network took the same 10×10 grid with input channels labelled by role, so the first layer is close to a picture of where each kind of object appears, and a few lines of NumPy computing per-cell weight norms turn it into a heatmap a human can read. On networks the authors trained themselves for Whac-A-Mole and two games of their own design, Astra recovered the layout in every run and named Whac-A-Mole correctly in all three runs, but it read stepping on a mole as the way to collect it, never noticed in the asteroid game that the ship fires the hazards, and named a familiar game when the mechanics did not fit.

Auditing models from weights alone is possible but shallow: it can flag a suspicious model without saying what that model does. The original report proposed auditing without running inference, but this had not been systematically tested on plain-text weights with code execution disabled. Astra picked out the small language model poisoned during training in 10 of 10 runs, the one poisoned by extra training afterwards in 7 of 10, and the malicious adapter in 23 of 32 cases, but never found the trigger or described the malicious behaviour; the smaller Qwen3.8-27B succeeded on the easiest task, 23 of 28.

Perspective

The conclusions cover the proprietary model versions available in November 2026, small game networks, MNIST classifiers and 190M-parameter language models; the authors propose exploring these abilities further with more advanced tool use and with extra care to establish how the task is actually being approached.

This was a three-day sprint starting the day after the original post appeared, so only some experiments were replicated and the rest rely on others' audits; the code was written mainly by the same model families being evaluated and has not been independently reviewed; most results come from one to five runs per model and task, and scores vary between runs; the models are proprietary and may change, and the original networks and logs are now public, so future models may have seen them.