What self-improvement means here
According to the Last Week in AI channel, recursive self-improvement means an AI system improves not only its answers but also the process it uses to become better. Andrey Kurenkov and Jeremie Harris discuss two research papers at the end of episode 259: The Last AI Built by Humans and Dream-RSI. The first provides a map of the field; the second presents a method for improving exploration.
The channel describes today’s coding agents as an example of a first step: AI helps researchers execute work faster. A system that chooses its own improvement strategy, gathers useful experience and changes its learning environment would be taking on further parts of that process.
Five levels in the research map
The first paper organizes autonomy into five levels. L1 is executing improvement work. L2 is choosing improvement strategies. L3 is acquiring useful experience or learning signals. L4 is adapting the environment used for improvement. L5 is improving the improvement mechanism itself.
Kurenkov and Harris use this classification to distinguish a better prompt or tool setup from deeper changes in how a system learns. The discussion also covers the practical work behind reinforcement-learning environments and the different objectives used during initial training and later training. Compute efficiency is one recurring objective; designing useful feedback is another.
How Dream-RSI reuses experience
Dream-RSI starts with an exploration policy: a method for choosing what to try next. A coding agent explores a task and records the paths it took in a discovery tree. That history becomes a replay simulator. Other exploration policies can then be tested against the recorded paths without rerunning every expensive experiment.
The channel explains the trade-off through the branches of this tree. Repeating all exploration would cost a lot, while only memorizing a known answer gives little room to adapt. Replaying the explored part of the tree lets the method compare alternative decisions while reusing the work already done.
The loop continues
After comparing candidates in replay, the improved policy goes back into live exploration. It gathers new experience and creates another discovery tree. The pool of histories grows, giving the next round more material to learn from. This is the loop that the paper calls dreaming.
The project’s published examples include algorithm engineering, mathematical optimization and GPU kernel engineering. Its diagrams show how exploration, replay and policy updates fit together, and its comparison charts track discovery quality against computation. The podcast presents these papers as two complementary ways to understand progress in self-improving agents: defining the levels and building mechanisms for moving between them.
