Google and Google DeepMind's Dream-RSI Let AI Improve Its Own Search Strategy

A new paper from researchers affiliated with Google Research, Google DeepMind, and several academic partners proposes a less familiar route to recursive self-improvement. Instead of asking an AI model to train its successor or modify its own weights, Dream-RSI optimizes how an agent explores a problem.

The Dream-RSI paper, submitted on September 16, 2026, is titled Recursive Self-Improvement through Evolving Worlds. Its central insight is that an agent’s exploration history can become a replay environment for testing better search strategies. A costly real-world run produces a tree containing attempted solutions, evaluated results, failures, branching decisions, temperatures, and compute allocation. Dream-RSI then reuses that tree to evaluate alternative exploration policies without calling the coding agent or evaluator again.

The underlying language model remains fixed. What changes is the machinery around it: which branch to expand, how many candidates to generate, how broadly to search, and when to stop. That makes Dream-RSI narrower than the usual vision of self-modifying AI, but also considerably more practical.

Self-Improvement Moves Above the Model Weights

Recursive self-improvement, or RSI, is often framed as a model changing its own architecture, training process, code, or parameters to produce a more capable successor. Dream-RSI moves the optimization target up a layer.

Its coding model does not update its weights. Instead, the system evolves a separate exploration policy that decides how the fixed model should spend its attempts. The policy can:

• Select an existing solution to expand.
• Generate groups of one, two, four, or eight candidates.
• Choose among several sampling temperatures.
• End the exploration process early.

The paper implements this controller as a compact neural network with roughly 30,000 parameters. It encodes the current discovery tree, uses attention to reason across its nodes, and produces probabilities for the next exploration operation. The researchers optimize this policy with an evolutionary strategy rather than gradient-based training.

Calling this “harness optimization” is useful, provided the term is not stretched too far. Dream-RSI does not let an agent arbitrarily rewrite its tools, system prompt, evaluator, or runtime. It learns within a constrained action space designed by the researchers. Even so, those actions cover several consequential decisions that engineers normally tune by hand.

How Dream-RSI Closes the Self-Improvement Loop

Dream-RSI starts with the observation that long-running coding and scientific discovery agents already produce structured data. Each candidate is derived from a parent solution, tested by an evaluator, and assigned a metric. Over time, these relationships form a discovery tree.

The system turns that by-product into a recursive process:

1. Explore online. A fixed AI model generates candidate programs or formulas. The evaluator runs each candidate and records its result in a discovery tree.
2. Construct replay simulators. Completed trees become environments in which other exploration policies can traverse the already computed branches in different orders.
3. Evolve the exploration policy. Candidate policies are compared inside a pool of replay simulators. Better-performing policies are selected, combined, and mutated.
4. Return online. The winning policy controls the next real exploration run. That run produces another discovery tree, which is added to the simulator pool.

The next policy is therefore optimized against a larger and more varied collection of histories than the previous one. The loop becomes: explore, record, replay, improve, and explore again. This recursive structure, rather than weight modification, is the basis for the paper’s RSI claim.

Dream-RSI does require a starting point. The researchers first build a bootstrap discovery tree using a fixed exploration strategy, extract successful trajectories, and use them to initialize the small policy network through imitation learning. Recursive evolution begins after that initialization.

The Replay World Is Exact but Strictly Bounded

The word “dreaming” could suggest that Dream-RSI trains a generative world model to predict what would happen under new decisions. It does not.

The replay simulator returns outcomes that have already been observed. If a candidate policy selects a stored branch, changes the order of exploration, uses a different grouping, or stops earlier, the simulator can score that trajectory without rerunning the underlying program. There is no learned approximation of the evaluator and no synthetic prediction of an untested program’s performance.

That gives Dream-RSI an important reliability advantage. Within the recorded tree, the results are grounded in actual executions rather than a model’s estimate of those executions.

It also imposes a hard limitation. Dream-RSI cannot use replay to discover the result of an algorithm that was never generated. It cannot evaluate branches that do not exist in its accumulated histories, and it cannot extrapolate beyond the stored outcomes. Fresh discoveries still require an online round involving the coding model and evaluator.

The paper’s “zero execution cost” claim should be read in that context. Comparing policies in replay requires no new coding-agent or evaluator calls, but the original discovery tree was expensive to produce, and evolving the policy still consumes ordinary computation. Dream-RSI amortizes execution cost rather than eliminating it.

GPU Kernel Results Reward Better Compute Allocation

The clearest results come from four GPU kernel engineering tasks drawn from KernelBench. The authors compare Dream-RSI with a recursive baseline that retains the same fixed exploration policy across rounds.

They report that Dream-RSI:

• Matched VGG16 performance with 2.43 times fewer generations.
• Matched LayerNorm performance with 1.79 times fewer generations.
• Achieved 2.09 times higher performance on ConvDiv at a comparable budget.
• Achieved 1.44 times higher performance on ConvMax at a comparable budget.

For VGG16, Dream-RSI reached the fixed policy’s best result after 61 generations rather than 148. LayerNorm required 61 generations rather than 109. On ConvDiv and ConvMax, both methods received 151 generations, allowing a more direct comparison of final performance.

The policy did not simply become greedier with each round. In one analysis, it reduced the average number of evaluated attempts from about 110 to 50 as results improved. When progress stalled, it widened exploration again. Those later increases in compute coincided with new performance gains, suggesting that the policy was learning when additional breadth was worth paying for.

Results outside GPU kernels were also positive, although not uniformly dominant. At a fixed 10,000-generation budget on LassoBench, Dream-RSI improved the average score over recursive fixed exploration with both Gemini backbones tested. On three formula-discovery problems, it produced the best reported Sum Difference result and tied the best Circle Packing result, but did not lead the Auto Correlation task.

These remain author-reported experiments. They show that policy-level improvement can produce measurable gains across several structured discovery settings, but they are not yet independent replications.

Prompted Lessons Made Exploration Worse

One of the paper’s most revealing experiments compares replay-based policy improvement with a more conventional technique: summarize previous discoveries and insert those lessons into the coding agent’s prompt.

The researchers generated summaries from preceding rounds and supplied them as semantic guidance during the next ConvDiv exploration run. They tested this approach with both Dream-RSI and the fixed exploration baseline. In both cases, adding the guidance reduced performance and slowed convergence compared with the unguided version.