Simon Willison gave a language model on his laptop 17,253 decimal numbers and no explanation of what they meant. After 52 minutes of thinking, it answered: this network plays Breakout, 88% confident. It was right. That is weight-space programming in its simplest form: an AI model that reads, and sometimes rewrites, the raw weights of another neural network, without any training.
The term comes from MIT research that made the rounds this week. Large language models turn out to read small networks surprisingly well and can even write them from scratch. The headline claim, a small Qwen model that got six times better at maths, was mostly a fix for a looping problem, done with code and test runs. For European organisations there is a second story: an edit to 0.016% of a model’s weights changed its behaviour more than many fine-tunes do, and the yardstick Europe uses for model changes, training compute, barely registers it.
What is weight-space programming?
A neural network is a long list of numbers, its weights. Normally you only change them through training: show examples, measure the error and let an optimiser adjust the numbers step by step. Weight-space programming skips that. A language model receives the weights as text, reads them and edits them directly, guided by an instruction in plain language.
The term was coined by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL in their post Sparks of Weight-Space Programming of 2 November 2026. They ran their experiments with OpenAI’s GPT-6 Astra and call the results “sparks”: early signs, not a finished technique. There is no peer-reviewed paper yet, but the full logs are on GitHub.
How a language model reads a game from raw weights
In the first experiment, the researchers trained four small networks on simplified versions of Breakout, Pong, Pac-Man and Freeway. Each game runs on a 10 by 10 grid, and each cell has five channels: the player, moving objects, rewards, obstacles and the previous frame. That makes 500 inputs, followed by two layers of 32 neurons and five possible actions, or 17,253 weights in total.
GPT-6 Astra received those numbers as text, without the name of the game and without tools or code. It named the right game for all four networks.
How is that possible? Sean Goedecke explains it. The prompt tells the model which input belongs to which cell and channel. Look at how large the weights are per cell and you get a heatmap of the playing field. In the Breakout network, the bottom playing row (where the paddle moves) and the top rows (where the bricks sit) light up. We recomputed this from the published weights: the pattern is really there, as the animation above shows.
That makes the result easier to understand, not less impressive. An experienced ML researcher with pen, paper and patience would reach the same conclusion. The model does it inside its reasoning, over more than 110,000 characters of numbers.
A laptop model can do it too
Willison wanted to know whether this needs a frontier model. In his test of 3 November he used the exact prompts from the logs with his LLM command-line tool. Kimi K3.1 Pro identified Breakout in 9 minutes 41 seconds. He then tried Qwen 3.8 27B, an open-weight Alibaba model that runs locally in a quantised version of about 17 GB.
| Network | Qwen 3.8 27B’s answer | Confidence | Thinking time |
|---|---|---|---|
| Breakout | Breakout or an Arkanoid-like game | 88% | 52 min 25 s |
| Pong | Pong-like paddle and ball game | 75% | 1 h 4 min 54 s |
| Pac-Man | Pac-Man-style gridworld with an 8×8 maze | 72% | 46 min 12 s |
| Freeway | still running at publication | – | – |
Three out of three, for a model that fits on a well-specced laptop. Willison finds the reasoning traces fun to skim but almost incomprehensible. The point for Europe: reading weights is not a secret skill of one US lab. A strong open model you host yourself can work it out, given time. How much memory such a model needs is covered in our piece on local AI on the M6 MacBook Pro.
How much help the model got in each experiment
The research contains eight experiments, and that is where the debate lies. Read only the summary and it looks as if GPT-6 Astra did the same thing everywhere: look at numbers and understand them. In reality, the help it got differed a lot.
- Reading, without tools. Which game a network plays (4 out of 4), which shape a small diffusion model draws (3 out of 3, although Astra drew outlines where the model makes filled shapes), and repairing a network that could not get a car up a ramp. Astra changed 3.5% of the 171,664 weights in a single attempt, after which the car succeeded in all 1,024 test episodes.
- Writing, without tools or test runs. Astra wrote networks for three simplified Atari games (the MinAtar versions) from scratch, starting from all-zero weights. In Breakout it scored 517 points, against 25 for a network trained for 500 million steps with the standard PPO method. In Seaquest (53 against 89) and Asterix (53 against 55) it lost.
- Editing, with Python and test runs. A Go network of 1.48 million weights, the Qwen model and finding a sycophantic Gemma model. Here Astra could run code and execute the network up to 50 times. It could not find the sycophantic model from weights alone, in any of four attempts. Only with attention patterns did it succeed.
The Qwen jump from 7.9% to 48.1% was a loop fix
The most shared claim: Astra made Qwen3.5-0.8B much better at maths by editing 0.016% of its weights. MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval almost tripled.
The story behind those numbers is simpler. In thinking mode, the small Qwen model often gets stuck in a loop, something Qwen itself warns about on the model card. In the test, 93% of answers hit the 32,768-token limit, and a cut-off answer counts as wrong. Using Python, Astra wrote texts that did and did not loop, found one direction in the MLPs of layers 15 and 19 that carried the looping, and subtracted it from 64 columns per layer: 131,072 weights.
After the edit, 33.1% of answers still hit the limit. More finished answers means more correct answers. The researchers say so themselves: on Gemma 3 270M, which has no looping problem, the score only moved from 6.8% to at most 7.7%. Their conclusion is that such an edit can remove an obstacle but cannot add an ability the model lacked.
The technique is known as directional ablation, made famous by 2024 research showing that a single direction controls a chatbot’s refusals. Tools such as Heretic use it to strip safety refusals from open-weight models. The edited Qwen model is on Hugging Face under Apache 2.0, and an independent re-run on the full MATH-500 came out at 51.0%.
Why critics call it a magic trick
Matthew Honnibal, creator of spaCy, calls the presentation misleading. His comparison: stage magicians often perform the same trick in several ways, so the audience assumes a single explanation. The research groups three different methods under one name, and readers assume the Qwen gain also came from “just looking at the raw weights”. It did not: every experiment with a real language model used Python, and the Qwen result also used test runs. Honnibal does not dismiss the work. A year or two ago, models could not carry out this kind of ML task on their own.
Goedecke goes further: none of this is superhuman, and a competent researcher would have done exactly the same to the Qwen model. Forums such as Tildes and r/LocalLLaMA add two doubts. The reading task works because the prompt includes a map of the inputs, and what works on toy networks may not scale to models with billions of weights. The idea that this is recursive self-improvement does not hold up either: a big model repaired a small one, it did not improve itself. For the wider picture, see our piece on Google DeepMind’s pathways from AGI to ASI.
Why a 0.016% edit matters for Europe’s rulebook
The AI Act measures changes to a general-purpose AI model mostly in training compute. Under the Commission’s guidelines for general-purpose AI providers, someone who modifies or fine-tunes such a model becomes a provider only in exceptional cases, when the modification uses more than one third of the original model’s training compute. Most fine-tuning stays well below that bar.
Weight-space programming uses almost no training compute at all. The Qwen edit took a Python kernel without a GPU and 50 forward passes. Yet it nearly tripled the model’s instruction-following score and changed how it behaves on long answers. Directional ablation of the same kind can remove a model’s refusals. A compute yardstick sees none of that. It is a reasonable proxy for building a new model, and a weak one for changing what an existing model does.
This does not create a legal loophole for providers, whose obligations stay in place, and it is not legal advice. It does shift the risk to the organisations that deploy open-weight models. A model file can differ from the published one in two tensors and behave differently in ways a quick test will miss. The published Qwen edit shows good practice: its model card lists the sha256 hash of each changed tensor, so anyone can check exactly what was touched.
What European organisations can do now
- Pin and verify your weights. Record the hash of every open-weight model file you deploy, including quantised versions, and check it at load time. Treat a changed tensor like changed source code.
- Judge changes by behaviour, not by compute. Any edit to a model, however small, gets the same evaluation run as a new release: your own task tests, refusal and safety checks, and long-answer behaviour. Keep a change log with the reason for each edit.
- Use weight reading as a second look, not as an audit. A language model can inspect small in-house models for what they learned. The sycophancy experiment shows the limit: from weights alone it found nothing, so behavioural testing stays essential.
- Consider targeted repairs for small open models. If a small model you host keeps looping or shows one unwanted behaviour, finding and removing a direction can be cheaper than retraining. It remains expert work that you close with tests, ideally on your own sovereign stack, such as Mistral Large 4 for larger workloads.
- Reproduce before you believe. The prompts are public. Running the reading test on a local model takes an afternoon and shows your team what these claims are worth.
Ask what the model was given
A language model that finds meaning in 17,253 bare numbers is striking, and a laptop model doing the same is more striking still. Yet the most useful lesson sits in the fine print: every result depends on what the model received, whether a map of the inputs, a Python environment or fifty test runs. Willison draws the same line. He finds the research fascinating, but is not ready to declare source code obsolete or to believe the future belongs to incomprehensible machines editing their own weights. For European organisations, the practical question is the mirror image: if a model can be changed this cheaply, can you tell when yours has been?
Frequently asked questions about weight-space programming
Is weight-space programming the same as fine-tuning?
No. Fine-tuning changes weights through training: examples, a loss function and an optimiser. In weight-space programming a language model reads the weights and edits them directly, without gradients. In the experiments this worked for small networks and for one targeted repair, not as a replacement for training.
Does editing a model’s weights make me a provider under the AI Act?
Usually not. The Commission’s guidelines treat a downstream modifier as a new provider of a general-purpose AI model only when the modification uses more than one third of the original training compute. A weight edit like the Qwen fix uses a tiny fraction of that. Your obligations as a deployer, and the rules for high-risk systems, still apply, so check with your legal team.
Can ChatGPT or Claude read their own weights?
No. A chatbot has no access to its own weights. In the research, GPT-6 Astra received the weights of another, much smaller network as text in its prompt. A large language model has billions of weights, far too many for one prompt.
Can I try weight-space programming myself?
Yes. The prompts are in the public logs on GitHub. With the LLM tool and a local model such as Qwen 3.8 27B in LM Studio you can repeat the reading test, with a large context window and some patience. The edited Qwen3.5-0.8B model can be downloaded from Hugging Face.
