Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing.
Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlimited steps of course, because you have a finite amount of compute.
So each agent goes off, and by the end, Agent 1 got to 90% accuracy, Agent 2 got to 80% accuracy, and Agent 3 got to 89% accuracy. Agent 1 wins, of course.
But then you look at the refinement steps, and after 2 steps, Agent 1 was _already at_ 90% accuracy. So the agent spent the next 8 steps basically not moving at all. Agent 3 on the other hand, perhaps was continuously climbing in accuracy at every refinement step, but hit step 10 and had to stop.
Now because you recorded every step from every agent, you know what you'd do differently next time -- you'd not allocate as many steps to Agent 1, and give Agent 3 more steps, because perhaps that might result in Agent 3 coming up with a better answer.
From my understanding, that's what they built in the form of a "search" controller -- a way to evaluate automatically and reapply how you could allocate resources more effectively, when applied to a new problem.
But I guess my misunderstanding is how applicable the search controller is when applied to new problems -- just because one pathway stalled early for one problem, doesn't mean it would work for another?
Your understanding is basically correct.
"how applicable the search controller is when applied to new problems". We need meta-agent thinking pattern. Self-evolving agent has been very popular and we want to use agent to design a perfect agent. This is the problem that the "search" controller employed in this paper aims to solve.
Intuitively I wouldn't readjust how many steps they each do, but instead add another run afterwards, that get the same amount of steps as the previous, but now also with a concise description of what the previous attempts did and what they achieved, and ask it to improve. The amount of compute you have available, would dictate how many full iterations of this "fan out search > consolidate" workflow you can do.
In the paper (section 5.1), they actually tried to abstract high level directional insights into the prompt in order to see if that helped, and they basically found it underperformed a prompt that didn't have those insights at all, implying that directional guidance perhaps over-constrains things.
Could use the shorthand of 85%, 10%, and 5% as the way to divide the workload; 850/1000 computes, 100/1000, and 50/1000. Brute sprint, clean up & revaluation runs, then checksum and presentation.
Unless I'm misunderstanding, calling this RSI seems misleading?
This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.
Does RSI actually mean anything specific anymore? RSI, AGI, at this point seem like buzzwords. Sure AGI has definition that are measurable, say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI. We use AIs to train AIs which we use to train AIs, why is that not RSI? How much human intervention means that is not RSI?
So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes.
I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
RSI has always being a well defined name, and you can only have RSI if you have an intelligence capable or creating itself.
It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today.
It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
> say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI
What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
That's what 95% is intended to capture. That some level of expertise in an area is should be captured by 95% sample of the population, you could push it to 99% of 99.9%, but you want it to be quantified by a number to avoid arguments that something is not AGI because obscure field or expert exists that AI can not do.
95% better at 95% of the population is already approaching ASI. One could even argue that AGI is 50% better than 50% of the population.
All that said, this over rotation on benchmarking misses something critical. What is general intelligence? We assume that humans have it and we assume it is captured by benchmarks on "intellectual tasks", but it is probably the case that general intelligence is based displayed by judgement on uncertain outcomes. Benchmarks by their very nature have certain outcomes, they have a wrong and right answer.
Test AIs on questions we don't have the answers to and there is no clear right answer, but there will be at some point in the future. What will the economy do? Which US senators will be be re-elected that polling correctly suggests will not be re-elected. Which US senators, currently not in office, will actually pass bills representing the wishes of their voting base? What published papers will be seen are groundbreaking in 5, 10 15 years? What approach to unifying physics should be taken?
Agreed, what I understand from RSI would be models creating new models, or at least upgrading their own weights/architecture. It does not seem to be the case here.
According to industry leaders, we currently have AGI and RSI in the last month or so. Of course, we've seen zero evidence of any of this and have to take their word for it.
There is no way this could be reasonably framed as RSI.
This iterative, online optimization of an exploration policy is not recursively intelligent in any way. It simply reallocates the available computational resources to more promising (hopefully) parts of the search space as system conditions change over time.
The whole idea behind RL is that the agent improves over time by making actions in the environment and observing the next state and the reward then modifying its policy.
The exploration-exploitation dilemma still stands. "dreaming in the replay simulator" is jargon for the same replay ideas presented when Deep RL was first introduced (Q-learning with experience replay and all that).
FYI; the paper is clearly a reference to Danijar Hafner's 'Dreamer' line of work, which was published in 2019, and which Danijar has continued to iterate on. https://arxiv.org/abs/1912.01603
Is this a good place to ask why no one seems to be worried that recursive self improvement might be dangerous? To me that seems like a really bad idea but I’m interested to hear the pro-RSI side of things.
The replay simulator from history for off-policy eval is clever - avoids expensive rollouts. Curious how they prevent the policy from overfitting to already-discovered branches and going stale as the search space expands?
This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.
It says to read the complete history. Would that be analogous to reading all of one's chat threads, or just the history of the relevant chat thread that it's a part of?
That seems to refer specifically to a provided history for it to read - presumably, this would just have the project it's running in. ("Variables (‘$node_dir‘, ‘$history_dir‘, ‘$baseline_dir‘, ‘$eval_program‘, ‘$problem_file‘) are filled in by the calling system.", meaning that a history directory for it to read is provided by the harness.)
Fairly certain all the labs are doing this (RSI) at this point. It's a question of how public their proclamations are about it and how they're positioning PR etc.
Even today's lighter weight models know how to write kernels and optimize them. I've had DeepSeek 4.1 Flash tune the crap out custom CUDA kernels on my own codebase and it was entirely competent at it. And cheap.
The innovation pieces will be in the harnesses to support this. Which I guess is partially what's going on here.
It's not really RSI if you are just using the AI as a tool to help make it better. It has to be doing it itself, no? Otherwise self-hosted compilers are RSI.
"Across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings."
Anyone who seriously believe that and isn’t being hyperbolic for shock purpose is delusional. It’s a pretty standard research paper, you cannot assume any research that contains the terms RSI to be revolutionary
All along I had thought that "AGI", "RSI", etc. were at the model level: but this paper seems to be talking about "agents", etc. I'm not sure having a swarm of agents explore a problem space in parallel via brute force is what "AGI" is about. I'd be happy to be proven wrong.
AGI and RSI are both meaningless terms, meaning whatever you choose them to mean.
RSI is the new sexy. Models are RSI-ing themselves towards the singularity, these folks' agents are RSI-ing themselves towards mastery of their training environments, and my pet cat is RSI-ing himself into the best cat that he can be.
RSI has to be a harness because recursion is a loop, compute can be amortised by distilling behaviour into weights but the breakthrough would come from a harness. And agree, what Dream-RSI describes is a lot closer to online policy optimisation but pre-existence of a primitive doesn't have to detract from novel application.
I ended up building a simplified version of this as /self-improve in https://github.com/DanMcInerney/orchflows. History is the state ledger, memory and RSI just cite the history as evidence and can be rewritten. I feel like strong immutable state is the missing piece of the puzzle for most of these memory libraries.
Calling this paper "Dream" seems a bit speculative to me. The idea is interesting and reminded me somewhat of karpathy's work at https://github.com/karpathy/autoresearch
I might be wrong but is this really a solution for RSI? I interpret it's as a way to reducing wasted tokens and compute on paths that don't yield better results. It's an optimization. It's a faster way to get to RSI though. what's wrong?
Perhaps I'm not understanding it correctly, but here's my take on what the paper is doing.
Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlimited steps of course, because you have a finite amount of compute.
So each agent goes off, and by the end, Agent 1 got to 90% accuracy, Agent 2 got to 80% accuracy, and Agent 3 got to 89% accuracy. Agent 1 wins, of course.
But then you look at the refinement steps, and after 2 steps, Agent 1 was _already at_ 90% accuracy. So the agent spent the next 8 steps basically not moving at all. Agent 3 on the other hand, perhaps was continuously climbing in accuracy at every refinement step, but hit step 10 and had to stop.
Now because you recorded every step from every agent, you know what you'd do differently next time -- you'd not allocate as many steps to Agent 1, and give Agent 3 more steps, because perhaps that might result in Agent 3 coming up with a better answer.
From my understanding, that's what they built in the form of a "search" controller -- a way to evaluate automatically and reapply how you could allocate resources more effectively, when applied to a new problem.
But I guess my misunderstanding is how applicable the search controller is when applied to new problems -- just because one pathway stalled early for one problem, doesn't mean it would work for another?
Your understanding is basically correct. "how applicable the search controller is when applied to new problems". We need meta-agent thinking pattern. Self-evolving agent has been very popular and we want to use agent to design a perfect agent. This is the problem that the "search" controller employed in this paper aims to solve.
Intuitively I wouldn't readjust how many steps they each do, but instead add another run afterwards, that get the same amount of steps as the previous, but now also with a concise description of what the previous attempts did and what they achieved, and ask it to improve. The amount of compute you have available, would dictate how many full iterations of this "fan out search > consolidate" workflow you can do.
In the paper (section 5.1), they actually tried to abstract high level directional insights into the prompt in order to see if that helped, and they basically found it underperformed a prompt that didn't have those insights at all, implying that directional guidance perhaps over-constrains things.
Great explanation.
Do you think this could be extrapolated to areas with no objectively verifiable results / outcomes?
(Outside of math & science)
Could use the shorthand of 85%, 10%, and 5% as the way to divide the workload; 850/1000 computes, 100/1000, and 50/1000. Brute sprint, clean up & revaluation runs, then checksum and presentation.
Isn't that a challenge with RL anyway that for a lot of problems its hard to even know accuracy continuously for each step
[dead]
Unless I'm misunderstanding, calling this RSI seems misleading?
This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.
Does RSI actually mean anything specific anymore? RSI, AGI, at this point seem like buzzwords. Sure AGI has definition that are measurable, say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI. We use AIs to train AIs which we use to train AIs, why is that not RSI? How much human intervention means that is not RSI?
So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes.
I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
The systems that train them do.
RSI has always being a well defined name, and you can only have RSI if you have an intelligence capable or creating itself.
It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today.
It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
Clearly well-defined must live in a probability bracket. Only humans demand exactitude.
> say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI
What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
That's what 95% is intended to capture. That some level of expertise in an area is should be captured by 95% sample of the population, you could push it to 99% of 99.9%, but you want it to be quantified by a number to avoid arguments that something is not AGI because obscure field or expert exists that AI can not do.
95% better at 95% of the population is already approaching ASI. One could even argue that AGI is 50% better than 50% of the population.
All that said, this over rotation on benchmarking misses something critical. What is general intelligence? We assume that humans have it and we assume it is captured by benchmarks on "intellectual tasks", but it is probably the case that general intelligence is based displayed by judgement on uncertain outcomes. Benchmarks by their very nature have certain outcomes, they have a wrong and right answer.
Test AIs on questions we don't have the answers to and there is no clear right answer, but there will be at some point in the future. What will the economy do? Which US senators will be be re-elected that polling correctly suggests will not be re-elected. Which US senators, currently not in office, will actually pass bills representing the wishes of their voting base? What published papers will be seen are groundbreaking in 5, 10 15 years? What approach to unifying physics should be taken?
Yeah, this is absolutely not what anyone reasonable is thinking about when they say recursive self-improvement.
I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't groqued it fully yet.
Agreed, what I understand from RSI would be models creating new models, or at least upgrading their own weights/architecture. It does not seem to be the case here.
According to industry leaders, we currently have AGI and RSI in the last month or so. Of course, we've seen zero evidence of any of this and have to take their word for it.
This paper optimizes a controller/policy which will be used to agent itself in the next round
the use of the phrase here feels very clickbaity tbqh
There is no way this could be reasonably framed as RSI.
This iterative, online optimization of an exploration policy is not recursively intelligent in any way. It simply reallocates the available computational resources to more promising (hopefully) parts of the search space as system conditions change over time.
If this is RSI then all RL is RSI.
The whole idea behind RL is that the agent improves over time by making actions in the environment and observing the next state and the reward then modifying its policy.
The exploration-exploitation dilemma still stands. "dreaming in the replay simulator" is jargon for the same replay ideas presented when Deep RL was first introduced (Q-learning with experience replay and all that).
FYI; the paper is clearly a reference to Danijar Hafner's 'Dreamer' line of work, which was published in 2019, and which Danijar has continued to iterate on. https://arxiv.org/abs/1912.01603
The TalkRL podcasts on this line of work are reasonable accessible and quite interesting. https://www.talkrl.com/episodes/danijar-hafner https://www.talkrl.com/episodes/danijar-hafner-on-dreamer-v4...
Is this a good place to ask why no one seems to be worried that recursive self improvement might be dangerous? To me that seems like a really bad idea but I’m interested to hear the pro-RSI side of things.
The replay simulator from history for off-policy eval is clever - avoids expensive rollouts. Curious how they prevent the policy from overfitting to already-discovered branches and going stale as the search space expands?
This is a solid and very interesting paper! The authors were kind enough to publish the complete prompt for it (appendix B.1 on page 18), so anyone can try their approach with any LLM and see the results.
It says to read the complete history. Would that be analogous to reading all of one's chat threads, or just the history of the relevant chat thread that it's a part of?
That seems to refer specifically to a provided history for it to read - presumably, this would just have the project it's running in. ("Variables (‘$node_dir‘, ‘$history_dir‘, ‘$baseline_dir‘, ‘$eval_program‘, ‘$problem_file‘) are filled in by the calling system.", meaning that a history directory for it to read is provided by the harness.)
So its more like a harness (like Codex etc) on top of any existing LLM?
it's basically the kairos system of claude code harness.
[dead]
I don’t really understand this. Some real examples would help. Making a “simulator” out of a bunch of historic states does not tell me enough.
Would be very glad if anyone explained to me if and why this is revolutionary.
Fairly certain all the labs are doing this (RSI) at this point. It's a question of how public their proclamations are about it and how they're positioning PR etc.
Even today's lighter weight models know how to write kernels and optimize them. I've had DeepSeek 4.1 Flash tune the crap out custom CUDA kernels on my own codebase and it was entirely competent at it. And cheap.
The innovation pieces will be in the harnesses to support this. Which I guess is partially what's going on here.
It's not really RSI if you are just using the AI as a tool to help make it better. It has to be doing it itself, no? Otherwise self-hosted compilers are RSI.
That's my point about the harness.
Probably isn't by virtue of it being publicly released
Here you go:
"Across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings."
Why would you assume it is revolutionary?
They wrote “if and why”.
Because “recursive self-improvement” is widely understood, even at labs, as having a non-trivial chance of eradicating human life.
This is like asking why a paper called “I discovered God. This is the specific proof.” is revolutionary. Just tedious levels of incredulity.
Anyone who seriously believe that and isn’t being hyperbolic for shock purpose is delusional. It’s a pretty standard research paper, you cannot assume any research that contains the terms RSI to be revolutionary
All along I had thought that "AGI", "RSI", etc. were at the model level: but this paper seems to be talking about "agents", etc. I'm not sure having a swarm of agents explore a problem space in parallel via brute force is what "AGI" is about. I'd be happy to be proven wrong.
AGI and RSI are both meaningless terms, meaning whatever you choose them to mean.
RSI is the new sexy. Models are RSI-ing themselves towards the singularity, these folks' agents are RSI-ing themselves towards mastery of their training environments, and my pet cat is RSI-ing himself into the best cat that he can be.
RSI has to be a harness because recursion is a loop, compute can be amortised by distilling behaviour into weights but the breakthrough would come from a harness. And agree, what Dream-RSI describes is a lot closer to online policy optimisation but pre-existence of a primitive doesn't have to detract from novel application.
I ended up building a simplified version of this as /self-improve in https://github.com/DanMcInerney/orchflows. History is the state ledger, memory and RSI just cite the history as evidence and can be rewritten. I feel like strong immutable state is the missing piece of the puzzle for most of these memory libraries.
Calling this paper "Dream" seems a bit speculative to me. The idea is interesting and reminded me somewhat of karpathy's work at https://github.com/karpathy/autoresearch
[flagged]
I might be wrong but is this really a solution for RSI? I interpret it's as a way to reducing wasted tokens and compute on paths that don't yield better results. It's an optimization. It's a faster way to get to RSI though. what's wrong?
Hapsburgs: Recursive self improvement by inbreeding
Have Sutton and Barto chimed in on "RSI" yet? Does anyone know?
this is just not good science or engineering. ok as a student project i guess.
ctrl+f 'safe'
ctrl+f 'danger'
Yup, we're all gonna die :(
[dead]
[flagged]
[dead]
[dead]