Asked to hand-write weights, frontier LLMs skip the Transformer: a sparse fact table beats the same model's written Transformer on ARC with about 1,800 fewer parameters
Given no gradient training, ten labeled examples and a sandbox, three frontier LLMs asked to design their own target model all abandoned the Transformer and instead wrote plain-text facts into a sparse table read by a fixed procedure. Claude Opus 5.5's table model scored higher ARC multiple-choice accuracy than the most accurate Transformer the same LLM wrote, using about 1,800 fewer parameters; the authors then propose an associative-memory model that reads a fixed number of rows and runs up to 17 times faster on a GPU than a same-sized Transformer.