🔍 Read the full analysis: AI Reads Model Weights: The Ideas And Limits Of Weight-Space Programming on ThorstenMeyerAI.com
Turn your quiet moments into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
TL;DR
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra could infer functions from small neural-network weights, repair selected failures and create networks in eight experiments. The results include a reproduced Qwen maths improvement, but the work is a blog post rather than a peer-reviewed paper, and the experiments are limited in scope.
Four MIT CSAIL researchers report that OpenAI’s GPT-6 Astra identified, edited and wrote neural networks by working directly with their numerical weights, rather than using conventional training. Their November 2 blog post documents eight experiments, including a repair to a looping language model and a car-control network that learned to reverse, while also showing cases where the approach did not help.
The researchers call the method weight-space programming: a general-purpose language model examines or changes a neural network’s weights, the numerical values that shape its computations. The team says Astra did not use gradient descent or an optimizer to make the edits. In some tests it received weights as text and answered directly; in others, researchers ran Python code it wrote inside a restricted sandbox. The released logs reportedly show no built-in tools such as shell access or web search were available to Astra.
In one repair test, a car-control network had recorded zero successful reversals in 256 episodes before Astra changed 3.5% of its weights. Afterward, it succeeded in 1,024 of 1,024 episodes, according to the post. In another test, an edit affecting 0.016% of Qwen3.5-0.8B’s weights coincided with its MATH-500 score rising from 7.9% to 48.1%. A Hugging Face engineer later rebuilt that edited model from the released logs and reported a score of 51.0%.
The post also describes Astra identifying the games represented by four small networks and recognizing patterns in three tiny language models. For generation, the researchers say it wrote small game-playing networks and a 1.5-million-weight network for 9×9 Go from zero. In 50 test runs, the Go network beat a same-size comparison network trained on about 20.5 million games. These are results reported by the authors, not independent evaluations across a broad range of models.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
Direct Edits to Model Weights
If the results hold up, weight-space programming could offer a different way to inspect and alter small models: a language model might identify a recognizable computation or make a targeted correction without retraining the entire network. That possibility could interest researchers and companies maintaining small, open models, especially when a specific failure can be tested against a clear benchmark.
The experiments do not show that AI can reliably repair arbitrary systems or replace training. The reported gains concern particular networks and tasks, and an improved benchmark score does not by itself establish that a model is safer, more capable in general or dependable in real-world use. For users, the findings do not change how AI products work today. For model developers, they raise a practical question about how weights are inspected, edited and validated.
Products Worth Considering
Therabody Theragun Prime 6th Generation Deep Tissue Massage Gun, Black
BLUEAIR Mini Restful Air Purifier with Sunrise Alarm Clock and Nightlight
Therabody Theragun Mini 3rd Generation Massage Gun, Black
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, Andrej Karpathy described neural networks as “Software 2.0”: programs produced through optimization and represented in their weights, rather than written line by line by people. He also observed that directly coding in millions of weights is difficult. The MIT group’s experiments revisit that idea by asking whether a language model can reason about weights as a kind of program.
The researchers are Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab; the post says Takehi and Teoh contributed equally. The source describes the work as a blog post with public logs, not a peer-reviewed paper, and says it was not posted on arXiv. It reports one run per setting, small networks and a single model family per task, limits that matter when judging how widely the results may apply.
““Software 2.0””
— Andrej Karpathy, describing neural networks in 2017
How Far the Results Generalize
The post does not establish whether the results will generalize to larger models, different architectures or other tasks. The source describes one run per setting and a limited set of small networks; repeated trials and independent replication would help establish how stable the reported outcomes are. The Qwen result has a reported external rebuild and re-test, but that does not validate every experiment.
It also remains unclear how reliably a model can distinguish meaningful internal structure from patterns that are merely easy to interpret after the fact. The source notes that the visual evidence behind one game identification could also be reproduced with a short Python script. The researchers further report that Astra could not distinguish an honest model from a sycophantic one based on weights alone. The post’s claims are therefore evidence of a promising experimental technique, not proof of a general-purpose model-auditing or repair system.
Replication and Broader Testing
The immediate next step is independent replication across more models, tasks and repeated runs, with evaluation that checks not only benchmark scores but also unintended changes. The researchers’ released logs give other teams material to examine, but the source does not announce a peer-reviewed publication or a schedule for further testing.
Until such evidence is available, weight-space programming remains a research result rather than a demonstrated production tool. Developers considering it would need to verify each edit against the original failure and test for regressions. Whether language models can apply the approach consistently beyond the small experimental networks described in the post remains open.
Key Questions
What is weight-space programming?
It is the approach tested by the MIT team in which a language model reads or changes another neural network’s weights directly, rather than relying on conventional gradient-based training for that operation.
What did Astra reportedly repair?
The post describes a car-control network that went from zero successful reversals in 256 episodes to 1,024 successes in 1,024 episodes after an edit. It also reports a large MATH-500 score increase after a small edit to Qwen3.5-0.8B.
Did the approach work in every test?
No. The researchers say edits did not improve maths on Gemma 3 270M, which was not looping, and that Astra could not identify an honest model versus a sycophantic one from weights alone.
Has the work been peer-reviewed?
The source describes it as a blog post with public logs, not a peer-reviewed paper, and says it was not on arXiv at the time of publication.
Does this change how AI products work now?
No immediate product change is reported. The experiments suggest a possible research tool for targeted model inspection and editing, but the method’s reliability and broader applicability remain unestablished.
Source: ThorstenMeyerAI.com
