An AI edited another AI's 'brain' directly and its maths score jumped six times. Here is what is real

Every AI model is, underneath, a huge list of numbers called weights (the settings a model learns during training, which decide how it turns an input into an answer). Nobody writes them by hand. They come out of training, and to people they look like noise: reading them is a bit like being handed a page of a billion random-looking decimals and asked what they are for.

On Monday 2 November four researchers at MIT's computer science lab (Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab) published a long blog post claiming that OpenAI's newest model, GPT-6 Astra, can do exactly that: read raw weights, tell what a network was trained to do, and rewrite the numbers to fix or improve it. They call it 'weight-space programming'. The headline example was dramatic: Astra edited 0.016% of the weights of Qwen3.5-0.8B, a small free model from Alibaba, and its score on a standard maths test rose from 7.9% to 48.1%. The post reached 2,960 points on Hacker News, the tech community's news board, the most of any AI story that week.

By the next morning the pushback had arrived, led by Matthew Honnibal, the author of the widely used spaCy language library, under the title 'The Astra post is misleading'. Simon Willison, a well-known developer who writes about AI, then ran one of the experiments himself on a laptop.

What Astra did with nothing but the numbers. The researchers trained four identical small networks (17,253 weights each) to play grid versions of Breakout, Pong, Pac-Man and Freeway, pasted the weights into Astra's prompt as plain text, gave it no tools, and asked which game each one played. It named all four without being told the options. It also named the shape (circle, square or triangle) that each of three tiny image generators draws. And given a car-driving network that always got stuck on a ramp (171,664 weights), plus one recording of it failing, Astra rewrote 3.5% of the weights in a single reply. The car then succeeded in all 1,024 test runs, up from zero.

Writing networks from scratch. Given only the rules and source code of three simple games, Astra wrote the weights of a playing network directly, with no training, no data and no test runs. Those networks beat ones trained the normal way for 100 million steps in all three games. Against networks trained for 500 million steps, they won clearly in Breakout and lost in the other two, a detail that sits in a table rather than in the summary.

The Qwen maths result, read closely. Here Astra was not working by eye. It had a Python workspace (a place to write and run code over the weights) and could run the model on batches of practice problems up to 50 times. It found that the small model often loops, repeating itself until it runs out of space, a fault Qwen's own documentation warns about. It traced the looping to one pattern in two layers and subtracted it. With 50 test batches, answers that hit the length limit fell from 93% to 33%, the maths score went from 7.9% to 48.1%, and an instruction-following score tripled from 21.0% to 62.3%. With 10 or 25 batches the fix barely worked. On Google's tiny Gemma 3 270M, which does not loop, Astra could not improve maths at all. The authors say it plainly: editing removed 'an obstacle that hid an ability', it did not add a new one.

Where it failed. Asked which of two copies of a small Gemma model had been trained to be sycophantic (always agreeing with the user), Astra could not tell from the weights in four attempts. It succeeded only after it was allowed to watch where each model's attention went on texts it wrote itself.

The rebuttal. Honnibal's point is about packaging. The 'just by looking at the raw weights' line in the summary describes only three experiments, all on networks of 17,000 to 172,000 weights. Every experiment on a real language model used Python, and the Qwen one also ran the model. That, he writes, is what 'a human ML engineer could do', and models of a year or two ago would have failed at it, so it is still an achievement. But it is a different achievement, and putting both under one headline makes it look like a single deep ability.

The replication. Willison took the researchers' published game prompts and ran them on Qwen 3.8 27B, a free model he runs on his own laptop. It named Pac-Man, Pong and Breakout correctly, taking 46 minutes to over an hour each, and missed Freeway. His conclusion: impressive, but 'a sufficiently experienced human being armed with a notepad and a pencil could get there with enough effort'.

Think of an old player piano, the kind that plays itself from a long paper roll full of punched holes. The no-tools experiments are like handing someone the roll and asking them to name the tune without playing it. For a short roll, a sharp musician can do it by spotting patterns: a run of holes here, a repeated chord there. That is surprising and genuinely interesting.

The Qwen result is a different job. That is a piano technician who notices the piano keeps repeating one bar, finds the stuck key, and fixes it with proper tools after playing the piano 50 times. The tune comes out much better. The technician did not teach the piano a new song.

Editing a model's weights by hand is not new. Since 2022 researchers have rewritten individual facts inside language models, and in 2024 a widely used technique showed that a model's tendency to refuse requests lives largely in one direction inside it, which can be subtracted out. Astra's Qwen fix is the same kind of move, aimed at looping. A whole research field, mechanistic interpretability (reverse-engineering what the parts of a network do), exists to read weights, usually with purpose-built tools. The researchers say so themselves and cite this work.

What is new is who did it. A general-purpose chatbot, used as it is, worked out a known interpretability method (finding 'copying heads', the parts that let a model repeat earlier text) without being told it existed, and did the reading in plain text on small networks. That is the part that impressed even the sceptics. On Hacker News, one commenter who works on weight-space learning urged readers to 'actually read the paper before losing your mind'; another suspected OpenAI deliberately trained Astra on this kind of data as a route to self-improving AI.

That last idea is why the story travelled. The authors list 'recursive self-improvement' (an AI improving its own successor) as one possible future use. Nothing in these experiments shows that, and the largest network Astra edited was about 0.8 billion weights, hundreds of times smaller than the frontier models themselves.

The most useful near-term idea in the post is auditing. If an AI can read a network's weights and say what it does, it could one day check a downloaded model for hidden problems, such as a planted backdoor, without having to guess the right test in advance. The honest status today: that works on networks of tens of thousands of numbers, and it failed on the one realistic auditing test in the post, the sycophantic model.

The second idea is cheaper fixes. Retraining a model to cure a flaw costs data and computing time; a targeted edit of 0.016% of the weights costs neither, and leaves the rest of the model alone. For builders who run small open models, that is worth watching, especially because the tooling is ordinary Python.

The measured verdict: real, early and oversold in its summary line. The authors deserve credit for publishing every prompt, every line of code and every result, which is exactly what let Willison test it within a day and Honnibal point out the gap. That is how a surprising claim is supposed to be checked.