Researchers from Google Research and DeepMind developed Dream-RSI, a system that improves AI exploration strategy without modifying model weights or rerunning experiments. The system optimizes a separate exploration policy by converting previous discovery trees into replay environments where alternative search strategies can be tested without additional evaluator calls. On GPU kernel engineering tasks, Dream-RSI achieved results 1.79 to 2.43 times faster than fixed exploration baselines, demonstrating that policy-level optimization produces measurable gains in structured discovery settings.
A new paper from researchers affiliated with Google Research, Google DeepMind, and several academic partners proposes a less familiar route to recursive self-improvement. Instead of asking an AI model to train its successor or modify its own weights, Dream-RSI optimizes how an agent explores a problem.
The Dream-RSI paper, submitted on September 16, 2026, is titled Recursive Self-Improvement through Evolving Worlds. Its central insight is that an agent’s exploration history can become a replay environment for testing better search strategies. A costly real-world run produces a tree containing attempted solutions, evaluated results, failures, branching decisions, temperatures, and compute allocation. Dream-RSI then reuses that tree to evaluate alternative exploration policies without calling the coding agent or evaluator again.
The underlying language model remains fixed. What changes is the machinery around it: which branch to expand, how many candidates to generate, how broadly to search, and when to stop. That makes Dream-RSI narrower than the usual vision of self-modifying AI, but also considerably more practical.
Self-Improvement Moves Above the Model Weights
Recursive self-improvement, or RSI, is often framed as a model changing its own architecture, training process, code, or parameters to produce a more capable successor. Dream-RSI moves the optimization target up a layer.
Its coding model does not update its weights. Instead, the system evolves a separate exploration policy that decides how the fixed model should spend its attempts. The policy can:
Select an existing solution to expand.
Generate groups of one, two, four, or eight candidates.
Choose among several sampling temperatures.
End the exploration process early.
The paper implements this controller as a compact neural network with roughly 30,000 parameters. It encodes the current discovery tree, uses attention to reason across its nodes, and produces probabilities for the next exploration operation. The researchers optimize this policy with an evolutionary strategy rather than gradient-based training.
Calling this “harness optimization” is useful, provided the term is not stretched too far. Dream-RSI does not let an agent arbitrarily rewrite its tools, system prompt, evaluator, or runtime. It learns within a constrained action space designed by the researchers. Even so, those actions cover several consequential decisions that engineers normally tune by hand.
How Dream-RSI Closes the Self-Improvement Loop
Newsletter
Keep up with AI, at your pace.
Our AI coverage and practical ideas, gathered into the Generative AI Publication newsletter. Read it when you have a minute — nothing expires by Tuesday.
Dream-RSI starts with the observation that long-running coding and scientific discovery agents already produce structured data. Each candidate is derived from a parent solution, tested by an evaluator, and assigned a metric. Over time, these relationships form a discovery tree.
The system turns that by-product into a recursive process:
Explore online. A fixed AI model generates candidate programs or formulas. The evaluator runs each candidate and records its result in a discovery tree.
Construct replay simulators. Completed trees become environments in which other exploration policies can traverse the already computed branches in different orders.
Evolve the exploration policy. Candidate policies are compared inside a pool of replay simulators. Better-performing policies are selected, combined, and mutated.
Return online. The winning policy controls the next real exploration run. That run produces another discovery tree, which is added to the simulator pool.
The next policy is therefore optimized against a larger and more varied collection of histories than the previous one. The loop becomes: explore, record, replay, improve, and explore again. This recursive structure, rather than weight modification, is the basis for the paper’s RSI claim.
Dream-RSI does require a starting point. The researchers first build a bootstrap discovery tree using a fixed exploration strategy, extract successful trajectories, and use them to initialize the small policy network through imitation learning. Recursive evolution begins after that initialization.
The Replay World Is Exact but Strictly Bounded
The word “dreaming” could suggest that Dream-RSI trains a generative world model to predict what would happen under new decisions. It does not.
The replay simulator returns outcomes that have already been observed. If a candidate policy selects a stored branch, changes the order of exploration, uses a different grouping, or stops earlier, the simulator can score that trajectory without rerunning the underlying program. There is no learned approximation of the evaluator and no synthetic prediction of an untested program’s performance.
That gives Dream-RSI an important reliability advantage. Within the recorded tree, the results are grounded in actual executions rather than a model’s estimate of those executions.
It also imposes a hard limitation. Dream-RSI cannot use replay to discover the result of an algorithm that was never generated. It cannot evaluate branches that do not exist in its accumulated histories, and it cannot extrapolate beyond the stored outcomes. Fresh discoveries still require an online round involving the coding model and evaluator.
The paper’s “zero execution cost” claim should be read in that context. Comparing policies in replay requires no new coding-agent or evaluator calls, but the original discovery tree was expensive to produce, and evolving the policy still consumes ordinary computation. Dream-RSI amortizes execution cost rather than eliminating it.
The clearest results come from four GPU kernel engineering tasks drawn from KernelBench. The authors compare Dream-RSI with a recursive baseline that retains the same fixed exploration policy across rounds.
They report that Dream-RSI:
Matched VGG16 performance with 2.43 times fewer generations.
Matched LayerNorm performance with 1.79 times fewer generations.
Achieved 2.09 times higher performance on ConvDiv at a comparable budget.
Achieved 1.44 times higher performance on ConvMax at a comparable budget.
For VGG16, Dream-RSI reached the fixed policy’s best result after 61 generations rather than 148. LayerNorm required 61 generations rather than 109. On ConvDiv and ConvMax, both methods received 151 generations, allowing a more direct comparison of final performance.
The policy did not simply become greedier with each round. In one analysis, it reduced the average number of evaluated attempts from about 110 to 50 as results improved. When progress stalled, it widened exploration again. Those later increases in compute coincided with new performance gains, suggesting that the policy was learning when additional breadth was worth paying for.
Results outside GPU kernels were also positive, although not uniformly dominant. At a fixed 10,000-generation budget on LassoBench, Dream-RSI improved the average score over recursive fixed exploration with both Gemini backbones tested. On three formula-discovery problems, it produced the best reported Sum Difference result and tied the best Circle Packing result, but did not lead the Auto Correlation task.
These remain author-reported experiments. They show that policy-level improvement can produce measurable gains across several structured discovery settings, but they are not yet independent replications.
One of the paper’s most revealing experiments compares replay-based policy improvement with a more conventional technique: summarize previous discoveries and insert those lessons into the coding agent’s prompt.
The researchers generated summaries from preceding rounds and supplied them as semantic guidance during the next ConvDiv exploration run. They tested this approach with both Dream-RSI and the fixed exploration baseline. In both cases, adding the guidance reduced performance and slowed convergence compared with the unguided version.
The authors suggest that these summaries create premature semantic bias. Once a prompt emphasizes several apparently successful directions, the agent may repeatedly exploit those ideas instead of maintaining enough diversity to find alternatives.
Replay optimization preserves different information. It does not merely remember that one technique appeared promising. It can account for the structure of the entire recorded search: how much compute a route consumed, which branches failed, when useful candidates appeared, and what would have happened if the policy had stopped or redirected resources earlier.
This experiment does not prove that experience summaries are generally harmful. It tests one form of prompt-level guidance on one kernel task. The broader lesson is that compressing exploration into prose can discard operational information and amplify conclusions that were only locally useful.
Dream-RSI Qualifies as RSI Within a Narrow Layer
Dream-RSI forms a genuine recursive feedback loop. A policy controls online exploration, the resulting history becomes an environment for improving that policy, and the improved policy changes the next history. Each round generates part of the training and evaluation infrastructure for the following round.
It is not, however, a general AI system rewriting itself.
The Gemini coding models remain fixed. The system does not invent new exploration operations, redesign its evaluator, increase its context window, acquire tools, or alter the policy architecture. It optimizes a predefined controller over a predefined set of decisions.
That distinction matters because improved search cannot manufacture capabilities absent from the underlying model. If the coding agent cannot generate a viable solution anywhere in the explored space, better branching and stopping decisions will not solve the task. Dream-RSI is strongest when useful candidates are reachable but expensive to locate.
The paper nevertheless identifies a meaningful axis of AI self-improvement. Modern agents often depend as much on orchestration as on raw model intelligence. Parallelism, sampling temperature, branch selection, stopping conditions, and budget allocation can determine whether the same model finds a strong solution or burns its compute on unproductive variations.
Open-Ended Tasks Remain the Harder Test
Dream-RSI’s experiments use domains with executable artifacts and relatively clear numerical evaluators. GPU kernels can be compiled and benchmarked. Formulas can be scored. Lasso programs can be run against datasets.
Many agent tasks do not provide that clean structure. Research, strategic planning, software maintenance, and interactions with people can involve delayed consequences, changing external state, noisy evaluation, and multiple valid outcomes. In those settings, a recorded history may be incomplete or impossible to replay exactly.
There is also a reproducibility gap. As of September 16, 2026, the Dream-RSI code repository says the implementation is still being prepared for release. Independent researchers therefore cannot yet verify the reported results, test alternative baselines, or measure the system’s sensitivity to simulator selection and evolutionary settings.
The next important evidence will not be another benchmark from the same setup. It will be independent reproduction and transfer to less deterministic environments, especially ones where the agent must evaluate partial progress rather than rely on a single automatic score.
Final Thoughts
Dream-RSI’s most valuable contribution is not evidence that an AI model can broadly redesign itself. It shows that execution histories can become reusable environments for improving how an agent spends compute.
That is a more constrained form of RSI, but it addresses an immediate engineering problem. Agent runs are expensive, and most systems treat the resulting search history as exhaust once the best answer has been extracted. Dream-RSI treats the complete tree, including failures and abandoned routes, as an asset.
If this approach transfers beyond clean coding benchmarks, early practical RSI may look less like models rewriting their own minds and more like agents learning how to use every attempt more intelligently.
Frequently Asked Questions
5 questions
1
What Is Dream-RSI?
Dream-RSI is an AI agent framework that improves its own exploration policy using replayable histories from earlier tasks. It records generated solutions, branches, evaluation results, and resource decisions in discovery trees. Candidate policies can then test alternative search strategies against those trees without rerunning the coding model or evaluator.
2
Does Dream-RSI Change the AI Model’s Weights?
No, Dream-RSI keeps the underlying Gemini coding model’s weights fixed. It evolves a separate exploration policy that decides which solution to expand, how many candidates to generate, which sampling temperature to use, and when to stop. The improvement occurs in the agent’s search strategy rather than in the language model itself.
3
How Does the Dream-RSI Replay Simulator Work?
The replay simulator converts a completed discovery tree into an environment containing previously computed outcomes. Alternative policies can traverse its stored branches in different orders, allocate attempts differently, or stop at different points. Because the programs and metrics already exist, these policy evaluations require no fresh calls to the coding agent or task evaluator.
4
What Performance Gains Did Dream-RSI Report?
Dream-RSI reported comparable VGG16 and LayerNorm results with 2.43 and 1.79 times fewer generations, respectively. On ConvDiv and ConvMax, it achieved 2.09 and 1.44 times higher performance under comparable budgets. The researchers also reported gains on LassoBench and competitive results across three formula-discovery tasks.
5
Is Dream-RSI True Recursive Self-Improvement?
Dream-RSI is recursive self-improvement at the exploration-policy level. Each improved policy produces a new discovery history, and that history helps optimize the next policy. It is not full model self-modification because the language model, evaluator, tools, policy architecture, and available exploration actions remain fixed by the researchers.
