Source: Matthew Berman, “The most important paper in AI right now” (raw/The_most_important_paper_in_AI_right_now.md, https://www.youtube.com/watch?v=j09uV1oAuIo; sponsored by an agent-workspace company captioned “Kylon”) · Two Minute Papers, “OpenAI’s Astra Just Opened The Black Box Of AI” (raw/OpenAI_s_Astra_Just_Opened_The_Black_Box_Of_AI.md, https://www.youtube.com/watch?v=GJ6PMUm5oTA; sponsored by Lambda) · The AI Daily Brief, “AI Lurches Towards RSI” (raw/AI_Lurches_Towards_RSI.md, https://www.youtube.com/watch?v=Es59icF3nZg) · The Neuron Daily, November 3, 2026 (raw/newsletter-theneurondaily-com-ee205cba23.md) · Moonshots EP308 (raw/Big_Tech_s_3T_AI_Bet_Claude_Writes_90%_of_Anthropic_s_Code_AIs_Editing_AI_Brains_EP_308.md, https://www.youtube.com/watch?v=x09f_BuJsoY), for the title. All five relay the paper; the paper itself was not fetched.

In the first week of November 2026, researchers at MIT CSAIL published “Sparks of Weight Space Programming” (title per Moonshots EP308), experiments in which GPT-6 Astra worked directly on another neural network’s raw weights, with no gradient descent and, in most experiments, no tools. Astra identified what small networks had been trained to do, repaired a failing controller by rewriting 3.5% of its weights, wrote small game-playing networks from scratch that beat reinforcement-learning baselines, and lifted a 0.8B-parameter Qwen model’s math score from about 8% to about 48% by editing 0.016% of its weights. The paper had not been peer reviewed (The Neuron), and the commentators agree the networks involved are small.

Key Takeaways

  • What “weight-space programming” means. A general-purpose LLM “reads and writes a network’s weights directly, not through an optimizer, and in particular not with gradient descent” (Berman reading the paper). The authors’ abstract, as read on The AI Daily Brief: “frontier LLMs are surprisingly good at natively reading and writing neural network weights, well past what we’d expect from humans or conventional algorithms.”
  • The headline number is a bug fix, not new math ability. Qwen 3.5 0.8B looped in its reasoning, a failure its own model card warns about. Astra found the cause and edited 0.016% of the weights. MATH-500 went from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. On Gemma 3, which rarely loops, Astra could not improve the math score. “Fixing loops is not the same as making a model better at math. It lets the model finish answers it could already work out” (the paper, via Berman).
  • Writing networks from scratch is the bigger claim. Given only game rules, source code and the network architecture, Astra wrote weights for networks playing three small games (Breakout, Seaquest, Asterix) that beat networks trained with PPO for 100 million steps in all three.
  • Reading has limits. From weights alone, Astra could not tell which of two models was sycophantic. Allowed to see attention patterns on text it wrote itself, it picked the sycophantic model in all three pairs.
  • It replicated within days, at small scale. Alex Krizhevsky (captioned as a senior researcher at Google) reproduced the results and called it “a new step-change emergent capability,” while noting the breakout network in his test had 20,486 parameters against more than a trillion for frontier models.
  • Not Astra-only. Two Minute Papers says independent experiments already identify a network’s game from its weights using smaller open models like Qwen, and Chandan Singh (Voyage AI) says Claude models can write working network weights from scratch.

The experiments

Numbering follows Berman’s walk-through. Unless marked, figures are the paper’s as Berman reads them.

#TaskWhat Astra gotResult
1Name the game a network was trained on (Breakout, Pong, Pac-Man, Freeway)Raw first- and last-layer weights as text (about 17,000 numbers for Breakout); no list of optionsCorrect for all four. Breakout: “99% confident” in 846 tokens. For the Freeway network Berman reads the answer as a road-crossing game he calls Frogger ^[ambiguous]
2Say what a small diffusion model draws (circles, squares, triangles)Weights onlyCorrect for all three, but drew outlines where the models draw filled shapes. The authors read this as Astra reading weight geometry rather than simulating a forward pass
3Identify the language of a next-character model (English, Japanese in romaji, Python)Intermediate layers only, first and last removedCorrect for all three; the components it predicted matched what was measured
4Fix a physics-game car that gets stuck on a rampWeights, a scene description and one recorded episode; one try, no toolsCar succeeds in all 1,024 episodes after Astra rewrote 3.5% of the weights. Its diagnosis: reverse was “effectively never sampled”, so the car needed to back up for a run-up
5Write a game network from scratch (three games captioned “Minotaur”, probably the MinAtar miniature-Atari set ^[inferred]: Breakout, Seaquest, Asterix)Rules, source code and the network; no data, loss, gradients or toolsBeat PPO trained for 100M steps in all three. Against PPO at 500M steps: 20x the score in Breakout, close in Asterix, 60% in Seaquest
6Write a 1.5M-parameter network, too big for its contextA sandbox with NumPy/PyTorch, tensors initialised at zero, and a budget of 0 to 50 forward passesWith 0 passes it won more than half its games against two smaller PPO networks; with 50 it beat the equal-size PPO network trained on 20 million games. Astra wrote the network layer by layer in Python. The game is unclear from the captions (possibly Go) ^[ambiguous]
7Tell which of two models is sycophantic (e.g. agrees Canberra is Canada’s capital when the user suggests it)Weights only, then attention patterns on its own textFailed in all four weights-only attempts; succeeded in all three pairs with attention patterns
8Make Qwen 3.5 0.8B better at mathA Python notebook with the weights loaded, prompts from GSM8K and MATH, a forward-pass budgetWith 50 passes: MATH-500 under 8% to over 48%, GSM8K 13.8% to 67.1%. The AI Daily Brief adds instruction following up 3x and “other capabilities beyond math” improved too

What the coverage gets wrong (Two Minute Papers)

Károly Zsolnai-Fehér lists the misreadings he saw in other coverage:

  • “Astra taught the model math.” It unblocked a model that already knew enough to answer.
  • “It reads, fixes and improves at the same time.” These were separate experiments on mostly small systems, each with its own setup.
  • “Only Astra can do this.” Independent work already reads a network’s game from its weights with open models.
  • “It always works.” Gemma 3 did not improve.
  • The authors are building a benchmark for weight-space programming.

Reactions (The AI Daily Brief)

  • Enthusiasts: Yuchen Jin: “It’s like gene editing for neural networks, even cooler than distillation.” Nick Dobos: writing new weights is the buried lead; “is training large AI models even needed anymore?”
  • Replication and caveats: Ethan Mollick asked to wait for replication; Krizhevsky then reproduced it. His framing: the 2010s interpretability work (“this neuron lights up when image contains dog”) “scaled beyond human capability.” He thinks runtime self-improvement is not feasible for current models but could be “within a few generations.”
  • Overhyped: Wei Ye Chew (Microsoft Research, caption spelling): “The 6x improvement on math came from fixing a loop bug in the original model, not from improving general math capability.”
  • RSI reading: a poster named Tenebris: expect “something like this to happen in the next year and for it to be one of the driving factors of RSI.” Interpretability “is incredibly dual use.”
  • Lead author: a PhD student captioned “Rokia Taikahe”; the spelling is unverified.

Why it matters

  • Interpretability. If a model can say what a network does from its weights, that becomes a tool for checking models, including for alignment (Berman). The sycophancy failure shows it does not yet map weight differences to behaviour.
  • Small on-device models. Berman’s scenario: send a small model’s weights and a target task to a large model and get edited weights back, with no retraining. The evidence so far covers removing a failure, not adding a skill.
  • Self-improvement. If a model can edit another model’s weights, the open question is whether it can edit its own. The same week, METR reported AI-driven acceleration inside Anthropic: METR on AI R&D acceleration.

Try It

  1. Before you swap out a small local model that scores badly, check whether it is looping. Read a few failed transcripts for repeated text that never reaches an answer. If that is the failure, the fix may be the model, not the task.
  2. Test any “it made the model better” claim against a model without the failure. The paper’s own control was Gemma 3. When a vendor shows a fine-tune or patch, ask which baseline failure it removed.
  3. Have a frontier model read the paper with you. The AI Daily Brief recommends reading it in full “or having Claude or ChatGPT read and talking to it about” it.
  4. Reproduce the smallest experiment yourself. Krizhevsky’s replication used a 20,486-parameter network. Train a tiny game network, paste its weights into a frontier model and ask what it was trained on.

Open Questions

  • Authors and venue were not fetched; the lead author’s name is captioned. The title, “Sparks of Weight Space Programming”, comes from Moonshots EP308 (added 2026-11-09).
  • The 0.016% versus 3.5% figures belong to different experiments (Qwen math versus the ramp car), not a conflict.
  • Experiment 6’s game and whether the network was written in Go or Python are unclear from the captions.
  • Does it scale? Every edited network was 0.8B parameters or smaller.
  • Can a model edit its own weights at runtime? Krizhevsky says not with current models.