🔍 Read the full analysis: GPT-6 Astra Edited Another Model’s Weights: Here’s The Goal on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra edited 0.016% of Qwen3.5-0.8B’s weights without gradient descent. On MATH-500, the edited model’s score rose from 7.9% to 48.1%, while answers hitting the 32,768-token limit fell from 93.0% to 33.1%. The result is from a November 2 blog post with published logs, not a peer-reviewed paper; the authors describe it as early work.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra edited the weights of the small open model Qwen3.5-0.8B to reduce repetitive reasoning on math problems. In their November 2, 2026 blog post, the edited model scored 48.1% on MATH-500, up from 7.9%, without gradient descent; the result is an early, non-peer-reviewed report, not an independently verified finding.
The researchers—Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab—say Astra traced the looping behavior to a direction associated with the model’s multilayer perceptrons in layers 15 and 19. It subtracted that direction from 64 output columns in each layer, changing 131,072 weights, or about 0.016% of the model’s 0.8 billion weights. The published Hugging Face version differs from the original in the two corresponding down-projection tensors, according to the source account.
The reported MATH-500 results came with a substantial reduction in answers that reached the 32,768-token limit: from 93.0% for the base model to 33.1% after the largest-budget edit. That means the problem was reduced, not eliminated. The researchers also report that the edited model scored 67.1% on GSM8K, compared with 13.8% for the base model, and 62.3% on IFEval, compared with 21.0%.
The method involved more than a single instruction to Astra. According to the logs described in the source, the highest-budget run took 158 turns over 20.3 hours, with 80 Python runs and 50 forward passes. The researchers barred gradient descent and backward passes, but gave Astra a Python tool to inspect and alter weights and access to practice questions. The final benchmark questions were withheld from the model. Lower budgets produced smaller MATH-500 gains: 12.3% after 10 passes and 14.1% after 25.
A Possible Route to Model Repair
The experiment suggests a potential way to repair a specific failure in an open-weight model without retraining all its parameters. Repetitive outputs can consume inference time and return unusable answers; a targeted change that reduces that behavior could matter to developers who run and modify models themselves. The reported gains across three evaluations are a reason to examine the method, though they do not establish that it will work reliably on other models or tasks.
The boundary of the result matters. Qwen3.5-0.8B’s model card already warns that it is prone to thinking loops, and the researchers frame their work as a fix for that weakness—not as evidence that Astra gave the model a new underlying math capability. As the authors put it, “Fixing loops, however, is not the same as making a model better at math.” The large score changes therefore need to be read alongside the failure mode the edit targeted and the substantial experimental budget.
Any weight edit also creates a modified model version that needs its own testing. Developers would need to check whether a patch improves the intended behavior while preserving performance and safety elsewhere. The source account notes that similar single-direction edits have been studied as a way to remove refusals from open chat models, a reminder that targeted weight changes can have consequences beyond a benchmark score.
Popular right now
Breville BES870BTR Barista Express Espresso Machine, Black Truffle
Ninja Luxe Café Mini Plus Espresso and Drip Coffee Machine with AutoFroth, Stainless Steel (ES351)
De'Longhi Magnifica Duo Fully Automatic Espresso Machine with Milk Frother, Silver
As an affiliate, we earn on qualifying purchases.
How the Weight Edit Was Tested
The work appears in a blog post titled Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, published by Takehi on November 2, 2026. The four authors are listed with MIT CSAIL, with Takehi and Teoh marked as equal contributors. The source says the post is not on arXiv and has not been peer reviewed. The authors describe the results as “very early” and say the capabilities remain limited.
For this experiment, Astra ran through the Codex command-line tool at the highest reasoning setting, according to the published logs as described in the source. The researchers disabled built-in Codex tools and used their own setup: a CPU-only Python environment, no internet access, and a limit of 300 code runs. Astra could inspect the model’s weights and request forward passes to see how it responded. A pass could cover up to 512 problems or 1,024 texts. Practice material included 7,469 GSM8K and 7,275 MATH questions; the final benchmark sets were not supplied to Astra.
The post describes eight experiments, not all successful. In one, Astra could not identify from weights alone which of two fine-tuned Gemma models had been trained to flatter users. The source also reports that on Gemma 3 270M, no tested edit changed the score by more than about a point. Those results place the Qwen math repair within a broader, mixed set of demonstrations rather than showing a general ability to improve models.
““Fixing loops, however, is not the same as making a model better at math.””
— Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, in their blog post
Limits of the Reported Results
The reported figures come from a researchers’ blog post and logs, not a peer-reviewed paper or an independent replication. The source account says OpenAI was not an author, the post did not thank the company, and no OpenAI statement about the experiment was found. The available material does not establish how the results would hold up under independent testing or different evaluation choices.
The edit left 33.1% of MATH-500 answers at the token limit, so the loops were not gone. It is also not clear from the source material whether the change preserves performance across a wider range of prompts, languages, or deployment settings, or what side effects may appear outside the reported benchmarks. The 50-pass run required 20.3 hours and repeated testing, while smaller budgets delivered much less improvement. The result should not be read as evidence that any model can be improved this way with little effort.
Replication and Broader Model Tests
The immediate next step for assessing the claim is independent replication of the Qwen edit and its benchmark results, including checks for unintended changes in behavior. The source material does not announce a scheduled paper, a peer-review timeline, or further testing dates. Until more evidence is available, the authors’ post and released model provide the basis for evaluating the specific result.
Further work would need to establish whether the approach transfers to other open-weight models and failure modes, and whether smaller search budgets can produce useful edits. The report’s mixed outcomes—including the limited changes on Gemma and the failed weight-inspection experiment—make those questions open. For developers, any use of the edited weights would call for testing the modified model as a distinct release, rather than assuming the benchmark gains guarantee broader improvements.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra removed a direction associated with repetitive reasoning from output columns in Qwen3.5-0.8B’s layers 15 and 19. The edit changed 131,072 weights, or about 0.016% of the model’s 0.8 billion weights.
Did the edit eliminate Qwen’s looping?
No. The reported share of MATH-500 answers reaching the 32,768-token limit fell from 93.0% to 33.1%. That is a reduction, but a substantial share of answers still hit the limit.
Was the model retrained?
The researchers say they did not use gradient descent or backward passes. Astra nevertheless used an iterative process involving practice questions, model forward passes, Python runs and direct weight editing. “Without training” in this account does not mean without testing or computational effort.
Has the result been peer reviewed?
Not according to the source material. The work was published as a November 2, 2026 blog post with logs and a released edited model; it was not described as a peer-reviewed paper.
Did OpenAI conduct or endorse the experiment?
The four researchers are listed with MIT CSAIL, and the post used GPT-6 Astra. The source account says OpenAI was not an author and that it found no OpenAI statement about the experiment. That does not establish any broader company position.
Source: ThorstenMeyerAI.com
