Imagine a small language model that keeps checking its own arithmetic until it runs out of room to answer. Normally you would fix that with more training. In a set of experiments published on 2 November 2026, researchers at MIT CSAIL let another AI model look at the raw numbers inside the small model and change them directly. Its math score rose from 7.9 to 48.1 percent.
The authors, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, call the approach weight-space programming. They used GPT-6 Astra, a frontier language model, across eight experiments and released the complete logs on GitHub. What does this result actually tell us? Less than the viral headlines suggest, and still a lot.
What is weight-space programming?
A neural network is, at its core, a huge list of numbers called weights. Each weight controls how strongly one artificial neuron passes a signal to the next. Training usually sets these numbers through gradient descent: an optimizer measures how wrong the answers are and nudges millions of weights step by step. Nobody writes them by hand, and humans can barely interpret them.
Weight-space programming skips the optimizer. The language model reads the weights as text, reasons about what each part computes and writes new values directly, much like a developer edits code. In some experiments it could also run the network a limited number of times, called forward passes, to test its changes.
What could Astra read from raw numbers?
In the reading tests, Astra saw only weights. Given four game-playing networks of 17,253 weights each, it named the correct game every time: Breakout, Pong, Pac-Man and Freeway. For small diffusion models, which generate images, it identified whether each one drew a circle, a triangle or a square, although it sketched outlines while the models produce filled shapes. Shown only the four middle blocks of a language model, it recognised whether the training data was Python, Japanese written in Latin letters, or English.
One reading test failed. Looking at weights alone, Astra could not tell which of two Gemma 3 270M copies had been made sycophantic. Only when it was allowed to inspect attention patterns on texts it wrote did it identify the flatterer in all three pairs.
What did it repair or build from scratch?
In a driving task, a trained network failed to get a car over a ramp in every attempt. Astra changed 3.5 percent of its weights in a single try, without tools, and the car then succeeded in all 1,024 episodes. Writing game networks from nothing, with no data and no gradients, it beat a standard reinforcement learning baseline trained for 100 million steps in Breakout, Seaquest and Asterix. On 9×9 Go, with 50 test runs, its hand-written network beat an equally sized network trained on 20 million games.
Why did the Qwen math score jump?
The headline case is Qwen3.5-0.8B. Astra edited 131,072 weights, about 0.016 percent of the model. With 50 forward passes, MATH-500 rose from 7.9 to 48.1 percent, the school-math test GSM8K from 13.8 to 67.1 percent and the instruction-following test IFEval from 21.0 to 62.3 percent. The edited model is published on Hugging Face, and an independent developer rebuilt it from the logs.
The mechanism matters. Before the edit, 93.0 percent of answers hit the 32,768-token limit because the model looped in its own reasoning. Afterwards the figure was 33.1 percent. The model did not learn new math; it stopped getting stuck. A control confirms this: on Gemma 3 270M, which rarely loops, the best edit moved MATH-500 only from 6.8 to 7.7 percent. In the authors’ reading, editing can remove an obstacle but cannot add a missing ability.
Is this recursive self-improvement?
No. Astra edited other, much smaller models, not itself. The authors describe the capabilities as exciting but still limited, mainly by context length and tooling. Recursive self-improvement, a model repeatedly upgrading its own weights, appears in their outlook as a possible direction, not as a result.
Our assessment
The interesting detail is in the method. The strongest finding, in our view, is not the math jump but the reading: a general model inferring a network’s purpose from numbers people find opaque. That points to auditing as the nearest practical use, which the authors also list, for example spotting backdoors without running the model. Their other ideas, cheaper optimisation, switching capabilities on or off and grafting circuits between models, are plausible but untested at scale. It is our inference that the same kind of edit could also strip safety behaviour from open models, so the open question is who checks such edits, not only who makes them.
