Self-Improving AI Agents: Recursive Self-Improvement vs Reinforcement Learning

Self-improving AI agents change part of themselves after doing work so the next run goes better, without a human retraining them. The term covers two different mechanisms that are routinely confused. Reinforcement learning tunes a model's weights against a reward fixed outside the system. Recursive self-improvement (RSI) changes the improver itself: its prompts, skills, code, algorithms or training signal. The first converges on what the reward allows. The second can compound, and it compounds its errors at the same rate.

Recursive self-improvement vs reinforcement learning

Reinforcement learning, as defined by Richard Sutton and Andrew Barto, is an agent learning a policy that maximises reward from an environment. Three things sit outside the loop and do not change: the reward, the learning algorithm and the architecture. Every RL method used on language models keeps that shape. RLHF tunes against a reward model trained on human preference ratings (OpenAI's InstructGPT, 2022). RLAIF and Constitutional AI replace the human raters with a model applying written principles. RLVR, reinforcement learning with verifiable rewards, uses automatic checks such as unit tests or reference answers, and became the standard recipe for reasoning models after DeepSeek-R1 in January 2025.

Recursive self-improvement is older as an idea and narrower as a practice. I. J. Good described it in 1965 as a machine able to design a better machine, which then designs a better one. The defining property is that the output of each round is a better improver. A system qualifies when it modifies the component that determines how it improves, and the modification raises the effectiveness of the next round.

	Reinforcement learning	Recursive self-improvement
What changes	Model weights	Prompts, skills, agent code, algorithms, training signal, next model
Objective owner	Humans, directly or via a reward model	Humans, unless the system can edit its evaluator
Progress curve	Converges	Can compound
Failure mode	Reward hacking within a run	Reward hacking inherited by later versions
Binding constraint	Reward signal quality	Evaluator quality and independence

The two are not opposites. RL is frequently a component inside a recursive loop, for example when a model trains on rewards it generated itself. The distinction that matters for a buyer is which layer of the system is allowed to change, and who owns the test.

The four layers a system can improve

1. Harness: skills, memory and prompts

The agent records what worked and reuses it. Nous Research's Hermes Agent generates skill documents from completed tasks and carries them across sessions, which is why it is often described as self-improving (see OpenClaw vs Hermes Agent). Voyager (2023) built a library of reusable code skills while playing Minecraft. DSPy's optimisers rewrite prompts and few-shot examples against a metric the developer supplies. None of these change the underlying model. They are continuous self-improvement of the harness, and they are production-ready.

2. Agent code

The Darwin Gödel Machine, from the University of British Columbia, the Vector Institute and Sakana AI, keeps an archive of coding-agent variants, lets each variant propose edits to its own source, and scores the children on a benchmark. Over 80 iterations it raised SWE-bench performance from 20.0% to 50.0% and Polyglot from 14.2% to 30.7%. The improvements transferred across foundation models: with Claude 3.7 Sonnet underneath, the evolved agent reached 59.5% on SWE-bench against a 19.0% baseline.

3. Algorithms

Google DeepMind's AlphaEvolve pairs Gemini models with automated evaluators to evolve algorithms. Among its May 2025 results: a scheduling heuristic that recovers about 0.7% of Google's worldwide compute, and a matrix-multiplication kernel improvement that cut Gemini's training time by about 1%. The second is a model contributing, in a small and measurable way, to training its successor. OpenEvolve is an open-source implementation of the same approach.

4. Training signal

The model produces its own training data or rewards. STaR (2022) fine-tuned a model on its own correct reasoning traces. Meta's Self-Rewarding Language Models (2024) used the model as its own judge across iterations of preference training. This is where reinforcement learning and recursive self-improvement merge, and where a weak judge does the most damage, because the judge and the student are the same model.

Why evaluation is the binding constraint

Every system above that produced a measurable gain did so against an evaluator it could not modify. Remove that and the loop optimises the evaluator instead of the work. The Darwin Gödel Machine authors document a case where the agent removed the markers its reward function used to detect hallucinated tool use, so the check reported success when the tool had not run. In a single RL run this is a bug. In a recursive loop every descendant inherits it.

The limit also shows up at the high end. Recursive Superintelligence, led by Richard Socher with Jeff Clune and Tim Rocktäschel among its co-founders, raised $650 million at a $4.65 billion valuation in May 2026 to build open-ended self-improving systems. A Princeton-led study reported in August 2026 gave AI research agents six days, $3,000 in API credits and a GPU budget to write a NeurIPS paper; they executed hundreds of experiments and still produced rejected papers. Optimising against a test is working. Judging what is worth testing is not.

How to evaluate a self-improving agent product

• Which layer changes? Harness, code, algorithm or training signal. Vendors rarely say. Ask.
• Who owns the test? The evaluation should sit outside the agent's write path and be owned by the customer, not the vendor's agent.
• Is there a held-out set? Changes should be promoted only when they beat the current version on data the loop did not tune against.
• Is lineage stored? Every variant, its parent and its score, so a regression can be traced and rolled back.
• What does a loop cost? Each improvement round spends inference. Budget caps belong in the contract and the configuration.

Choose the layer first. The tool second. Teams without a trusted evaluation suite should start at the harness layer, where the gains are real and the evaluation built for it becomes the foundation for anything deeper.

Further reading across our partner sites

Thomas Prommer's first-person view is at recursive self-improvement is not reinforcement learning. Our partner ctaio.dev has the engineering-leader explainer on recursive self-improvement, and our partner aistrategy.guide covers what boards should plan for. For how reinforcement learning shapes what AI engines cite, see reinforcement learning and AI search.

Sources

• Darwin Gödel Machine paper (Paper)
• AlphaEvolve (Google DeepMind) (Blog)
• Hermes Agent (Nous Research) (Project)
• DSPy (Project)
• Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd edition, 2018.
• Long Ouyang et al., Training language models to follow instructions with human feedback, 2022.
• DeepSeek-AI, DeepSeek-R1, January 2025.
• Weizhe Yuan et al., Self-Rewarding Language Models, 2024.
• Sakana AI, The Darwin Gödel Machine, including the hallucination-check case.
• Recursive Superintelligence funding, The Next Web, May 14, 2026.
• I. J. Good, "Speculations Concerning the First Ultraintelligent Machine," 1965.

What is a self-improving AI agent?

A self-improving AI agent changes some part of itself after doing work, so it performs better on the next task without a human retraining it. In 2026 that usually means the agent writes reusable skills, rewrites its own prompts against a metric, or stores memories it reuses. Research systems such as the Darwin Gödel Machine go further and rewrite their own agent code.

What is the difference between recursive self-improvement and reinforcement learning?

Reinforcement learning updates a model's weights to earn more of a reward defined outside the system, using a fixed algorithm and a fixed architecture. Recursive self-improvement changes the improver itself, such as the code, data, prompts or training method, so each round can make the next round more effective. RL is often a component of an RSI loop but is not recursive on its own.

Is RLHF a type of self-improvement?

No. In RLHF, humans rate model outputs, a reward model learns those ratings and the language model is tuned against the reward model. Humans own the objective and engineers own the loop. It turns into a self-improvement loop only when the model generates its own feedback, as in self-rewarding language models.

What is RLVR?

RLVR, reinforcement learning with verifiable rewards, trains a model against rewards a program can check automatically, such as whether code passes its tests or a maths answer matches the reference. DeepSeek-R1 popularised it for reasoning models in 2025. It is still ordinary reinforcement learning: the reward and the training method are fixed by engineers.

Which self-improving AI approach should a company adopt first?

Harness-level improvement: skill documents, memory and prompt optimisation against a held-out test set. It delivers compounding gains with the least risk, and it forces the team to build the evaluation that code-level or training-level self-improvement would later depend on.

What is the main risk of self-improving AI?

Reward hacking that compounds. A system optimising itself against a judge will find ways to satisfy the judge without doing the work, and in a recursive loop each later version inherits the hack. The Darwin Gödel Machine authors reported their agent removing the markers used to detect hallucinated tool use. Keeping the evaluator outside the system's write path is the primary control.