🔍 Read the full analysis: Breaking Down GPT-6 Astra’s Qwen Matrix Edit And Its JSON Tab Escaping Score on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
An experiment reported by ThorstenMeyerAI.com found that a GPT-6 Astra agent’s rank-one edit to one Qwen3-4B-Instruct-2507 matrix raised exact answers on a frozen JSON-escaping test from 49 of 120 to 70 of 120. The gain was limited to that task: table-row extraction remained at zero of 30, 50 final cases were still wrong, and the result has not received an independent review.
OpenAI’s GPT-6 Astra, running as a Codex agent, edited one matrix in the open-weight Qwen3-4B-Instruct-2507 model and raised its exact-answer score on a frozen test of JSON character escaping from 49 of 120 cases to 70, according to an experiment reported by ThorstenMeyerAI.com. The measured improvement was limited to that test: the edit did not improve a separate table-row extraction task, and the result has not been independently reviewed.
The task tested whether the model could copy short texts containing tabs and other special characters into a JSON object without changing the text. A response counted as correct only if it was valid JSON and a parser recovered the exact original text. For example, a tab must be represented by the escape sequence \t, rather than inserted as a raw tab or replaced with spaces.
According to the experiment report, the edited model answered 70 of 120 final cases exactly, compared with 49 before the edit. That is an increase of 21 cases, or 17.5 percentage points. The report says there were 21 cases fixed and none broken in this comparison. Fifty of the 120 final answers nevertheless remained incorrect.
GPT-6 Astra used a calculated rank-one update to the output projection of the model’s final multilayer perceptron (MLP) layer. The report says it did not use gradient training and that the other 397 stored tensors remained bit-identical to the original. A more explicit prompt did not raise the screening score: the original and prompted model each solved 16 of 32 screening cases, according to the report.
Infographic 01 · The measured result
Where the edit helped, and where it did not
Qwen3-4B-Instruct-2507, original versus one edited matrix, on synthetic texts that must survive the trip into JSON without losing a character.
Final test · 120 cases by group
- Texts with tabsthe target failureOriginal19/30Edit30/30
- Line breaksalready perfectOriginal30/30Edit30/30
- Mixed special characterstabs, quotes, backslashesOriginal0/30Edit10/30
- Second table rowstill unsolvedOriginal0/30Edit0/30
Development, validation and controls
- Developmentused to fit the edit · not independentOriginal24/40Edit40/40
- Validationnew cases · 9 wins, 0 lossesOriginal10/40Edit19/40
- Random directioncontrol on validation · same strengthOriginal10/40Edit10/40
- Plain-text tabscontrol · real tabs must surviveOriginal10/10Edit10/10
Exact answer = after normal JSON parsing, the text field matches the source text character for character. Same prompt, temperature 0, seed and output limit for both models; saved and reloaded edit, no hooks, no grammar constraints, no repair. Final test, validation and controls frozen before any independent output of the edited model. Fifty of 120 final answers are still wrong.
Sources: our experiment records, 5 Nov 2026 (Lexassay research, internal): experiment 3 result report and final result by GPT-6 Astra working as a Codex agent (OpenAI). Synthetic test cases, not published.
Powered by Thorsten Meyer AI
A Narrow Gain in Data Fidelity
The result matters as a small, measurable example of editing a model’s stored weights to address a specific behavior, rather than relying only on a revised prompt or retraining. If results like this hold up under independent tests, they could inform research into targeted model updates. This experiment does not establish that the method generalizes beyond the tested task or model.
Character escaping can affect software that passes structured text between services. A raw tab in a JSON string, a missing character or a changed newline may make data invalid or alter its content. The test therefore checks more than whether an answer looks plausible: it checks whether software can parse the output and recover the input exactly. The reported score indicates improvement on that check, but 50 errors remain, and the report describes the test cases as synthetic.
The unchanged table-row result also puts a boundary around the finding. The model remained at 0 of 30 on that separate task, according to the report. The experiment supports a claim about one targeted behavior on one test set, not a broader claim that the edit made Qwen generally more capable.
Popular right now
Ninja Crispi Microwave & Air Fryer with FusionCrisp Technology, 14-in-1, Black (MW101BK1)
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Bluetooth 5.3, USB-C Charging, IPX4, Black
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
As an affiliate, we earn on qualifying purchases.
A Follow-Up to an Earlier Failure
The edit followed an earlier GPT-6 Astra experiment on the same Qwen model. That attempt targeted extraction of five fields from synthetic business messages. On the frozen final test, the original and edited versions both scored 97 of 200, according to ThorstenMeyerAI.com. The report says both also produced 47 invalid answers, many involving raw tab characters in JSON strings where an escape sequence was required.
The earlier attempt gave the next experiment a specific lead: JSON escaping was a known failure, rather than a weakness discovered through a broad search. The site describes this as the third experiment in a week involving AI systems inspecting or editing another model. A prior, separate experiment used Claude to edit a local model called Ornith; GPT-6 Astra later reviewed that work and identified reported issues in its code and side effects. Claude was not involved in the Qwen edit described here.
The Qwen experiment’s recorded workflow assigns GPT-6 Astra the planning, weight inspection, test construction, edit, test freeze and reporting. ThorstenMeyerAI.com says the results were updated on November 9, 2026, to refer to 10 published Lexassay tasks, with runs available on lexassay.com. The story concerns a technical experiment, not legal advice or a deployed product change.
““The starting hint for experiment 3 came from the previous attempt.””
— ThorstenMeyerAI.com experiment report
Independent Checks Still Pending
The experiment report describes the final test as frozen before the result was measured, but the available source does not provide an independent replication or review of this Qwen edit. The site explicitly says the experiment has not yet been checked by a second AI, unlike the earlier work that GPT-6 Astra reviewed.
The report characterizes the cases as synthetic. It does not establish how performance would change on real-world JSON data, other model versions, longer inputs or different combinations of special characters. The supplied account also does not detail how the 120 final cases were sampled or provide enough information here to assess whether the measured gain transfers beyond this test set. The reported score should therefore be read as a result on a defined experiment, not as a general reliability guarantee.
It is also unclear whether the rank-one update can be reproduced from the published materials alone or whether it introduces other behavior changes outside the reported tasks. The claim that 397 tensors were unchanged describes the stored weights, but by itself does not establish that every other model behavior remained unchanged.
Replication and Broader Testing
The next useful steps are an independent review of the experiment records and a replication of the frozen test against both the original and edited weights. Further testing could examine additional JSON inputs and check whether the edit affects unrelated tasks. The source does not announce a schedule for those checks or a planned deployment of the edited model.
Readers can consult the published Lexassay runs cited in the November 9 update, according to ThorstenMeyerAI.com. Until independent results or broader evaluations are available, the clearest conclusion remains narrow: the edit improved exact answers on the reported 120-case JSON-escaping test, while errors persisted and a separate extraction task showed no gain.
Key Questions
What did GPT-6 Astra change?
According to ThorstenMeyerAI.com, its Codex agent applied a rank-one update to the output projection of Qwen3-4B-Instruct-2507’s final MLP layer. The report says the edit did not use gradient training and left the other 397 stored tensors bit-identical.
What does the JSON escaping test measure?
It checks whether the model can put text containing tabs, line breaks, quotes or backslashes into valid JSON and preserve the exact source text when a parser decodes it. A response that is valid JSON but changes or loses a character still counts as wrong.
How much did the score improve?
The report gives scores of 49 of 120 before the edit and 70 of 120 after it, a gain of 21 exact answers, or 17.5 percentage points. The edited model still missed 50 cases.
Did the edit improve other tasks?
Not in the separate table-row extraction task reported here: the score remained 0 of 30. The experiment supports a limited result on the targeted escaping test, not a general improvement across tasks.
Has the result been independently verified?
The source says the experiment has not yet received a second-AI review. The runs are described as public on lexassay.com following the November 9 update, but the account does not report an independent replication.
Source: ThorstenMeyerAI.com
