Maybe this is what the author is getting at, but if LP solvers are useful tools for editing weights, then Astra access to this methodology might make it more effective at weight editing.
The interpretation here is not so much that an LP can do Astra's work -- after all the Astra prompt was "improve Qwen's math by changing weights" or similar. Instead, it's that with some guidance (eliminate this behavior, preserve that behavior, as expressed through linear constraints) the solver is much more efficient than Astra's trial and error. So essentially the solver would be a way of cutting through all the time consuming stuff and expressing Astra's intent more immediately. And then presumably Astra could iterate on that to a better solution.
The use of an LP solver here is a really cool insight.
I don’t understand the negative reactions to your post, it’s a good insight.
Basically the weight edit approach is find a semantic direction for the undesirable behavior and steer away from it. With access to the weights I see no reason a model (even one as large as Astra) couldn’t apply that principle to itself. This probably includes at runtime if you wanted to give the model write access to itself
Which it did by doing an inefficient version of the aforementioned 1940s algorithm.
Essentially modern LLMs are able to do normal AI engineering tasks, like running optimization algorithms over weights. The media has been inaccurately describing this as if the AI had some special insight into the weights, when it reality it just performed the tasks an human engineer might do.
Right, humans don't need some special insight into weights either, since it's just a bunch of numbers that you can analyze like any other table with numbers. However, if you dump them on someone like in this prompt: https://github.com/RikiyaT/lm_understand-logs/blob/main/exp1... they might complain about the formatting or ask whether they can use Excel instead of doing calculations by hand. (The experiment with Qwen3.5-0.8B didn't use the "dump weights into the context as numbers" approach.)
Good article! The initial headlines from a few days ago made it seem like frontier models were becoming able to read, understand, and manipulate weights directly. This seemed unlikely to me since they aren’t trained on weights, they’re trained on tokens and task completion (RL). This analysis makes me think a better description is that Astra did some interesting model training research that included running small experiments with altering weights and found a nice edit that fixes some problems. Cool, also cool that this analysis finds a faster approach. But a far cry from the headlines when the paper first came out.
It's easy to trace that a bit sets a character in a file with absolute certainty; weights are in a highly dimensional space however, so even if ⍺v is good for one test it could easily regress for another.
I suppose the argument is that "editing weights" seems to imply a complex awareness of what each weight means, which is material for AGI hype. Really it's just doing very broad statistics, which is of course interesting but hardly unexpected.
Maybe this is what the author is getting at, but if LP solvers are useful tools for editing weights, then Astra access to this methodology might make it more effective at weight editing.
The interpretation here is not so much that an LP can do Astra's work -- after all the Astra prompt was "improve Qwen's math by changing weights" or similar. Instead, it's that with some guidance (eliminate this behavior, preserve that behavior, as expressed through linear constraints) the solver is much more efficient than Astra's trial and error. So essentially the solver would be a way of cutting through all the time consuming stuff and expressing Astra's intent more immediately. And then presumably Astra could iterate on that to a better solution.
The use of an LP solver here is a really cool insight.
Nice analysis. The simplicity of this indicates that it should be happening during training via regularization.
Does this describe a very powerful LLM, like Astra, editing smaller models?
Could Astra edit itself?
I don’t understand the negative reactions to your post, it’s a good insight.
Basically the weight edit approach is find a semantic direction for the undesirable behavior and steer away from it. With access to the weights I see no reason a model (even one as large as Astra) couldn’t apply that principle to itself. This probably includes at runtime if you wanted to give the model write access to itself
No, it describes a linear optimization algorithm from the 01940s editing a Qwen model with billions of parameters.
"In another experiment, they asked Astra to make Qwen3.5-0.8B better at math by editing its weights"
??
That's not what's being described here; that's described in another paper that this is a followup to.
Which it did by doing an inefficient version of the aforementioned 1940s algorithm.
Essentially modern LLMs are able to do normal AI engineering tasks, like running optimization algorithms over weights. The media has been inaccurately describing this as if the AI had some special insight into the weights, when it reality it just performed the tasks an human engineer might do.
Right, humans don't need some special insight into weights either, since it's just a bunch of numbers that you can analyze like any other table with numbers. However, if you dump them on someone like in this prompt: https://github.com/RikiyaT/lm_understand-logs/blob/main/exp1... they might complain about the formatting or ask whether they can use Excel instead of doing calculations by hand. (The experiment with Qwen3.5-0.8B didn't use the "dump weights into the context as numbers" approach.)
Good article! The initial headlines from a few days ago made it seem like frontier models were becoming able to read, understand, and manipulate weights directly. This seemed unlikely to me since they aren’t trained on weights, they’re trained on tokens and task completion (RL). This analysis makes me think a better description is that Astra did some interesting model training research that included running small experiments with altering weights and found a nice edit that fixes some problems. Cool, also cool that this analysis finds a faster approach. But a far cry from the headlines when the paper first came out.
It seems a little like splitting hairs.
Astra didn't directly edit the weights, it came up with a method to edit the weigths?
So, it used another tool to edit the weights?
This kind of seems like saying, I don't directly edit the bits on my hard drive, thus i'm not really the one saving a file?
It's easy to trace that a bit sets a character in a file with absolute certainty; weights are in a highly dimensional space however, so even if ⍺v is good for one test it could easily regress for another.
I suppose the argument is that "editing weights" seems to imply a complex awareness of what each weight means, which is material for AGI hype. Really it's just doing very broad statistics, which is of course interesting but hardly unexpected.
annnd we can just 'reinvent'(*) the same techniques used to uncensor models for some time.. one step at a time.
(*) scarequotes because the LLMs that suggested these approaches have presumably been exposed to the refusal direction suppression work