README.md

August 24, 2026 · View on GitHub

Awesome Self-Improving Modern Agentic Systems

A curated and continuously evolving resource hub for self-improving agentic systems
_{This repository brings together
📄 Papers · 📊 Benchmarks · ✍️ Blogs · 🎙️ Podcasts ·
💬 Interviews · 🎥 Videos · 🧑‍🏫 Workshops & Courses}
_{Contributions are welcome — open a PR to share relevant resources and help grow the community! 🌱}

This repository accompanies Self-Improvements in Modern Agentic Systems: A Survey .

🌐 Contents

Definition & Scope
Survey Paper
Taxonomy
Key Literature
- 🟦 Foundation Model Improvement
- 🟩 Scaffolding Improvement
Evaluation & Benchmarking
Related Resources
Contact & Community
Contribute
Citation

🧩 Definition & Scope

This repository focuses on:

Foundation-Model-Based Agents: Autonomous systems that use foundation models as cognitive cores and operate through persistent scaffolds such as prompts, memory, tools, and control logic.
Self-Improvement Loops: Agent-level update loops in which signals or artifacts produced through generation, intrinsic evaluation, or environment interaction are consolidated into persistent components of the agent.
Foundation Model Improvement: Parameter-level updates driven by intrinsic generative demonstrations, intrinsic evaluative feedback, or extrinsic exploratory experience.
Scaffolding Improvement: Persistent updates to prompts, memory, tools, workflows, or full agent scaffolds.

📄 Survey Paper

Main figure of the survey
Overview of self-improvement mechanisms in foundation model–based agentic systems.

🧭 Taxonomy

If you are not yet familiar with agents, this simple illustration shows how an FM-based agent differs from a standalone foundation model (e.g., an LLM).

View the animated comparison

An illustration of the distinction between a standalone foundation model and an FM-Based agent.

To ground our taxonomy, we first introduce a formal abstraction of a foundation model–based agent, which serves as the basic unit for all self-improving agentic systems considered in this survey.

Formal abstraction of an FM-based agent
Formal abstraction of a foundation model–based agent, consisting of a foundation model and its scaffolding.

Based on this abstraction, we categorize self-improving agents along two orthogonal axes, depending on which component is improved during learning and adaptation.

🚀 Self-Improving Agents
- 🧠 Foundation Model Improvement
  - 📄 1.1 Intrinsic Generative Demonstrations
  - ⚖️ 1.2 Intrinsic Evaluative Feedback
  - 🌍 1.3 Extrinsic Exploratory Experience
    - 🤖 1.3.1 Interaction with Grounded Task Environments
    - 🔮 1.3.2 Interaction with Simulated Proxy Environments
- 🏗️ Scaffolding Improvement
  - ✍️ 2.1 Prompt Optimization
  - 💾 2.2 Memory
  - 🛠️ 2.3 Tool
  - ♾️ 2.4 Full Scaffolding

📚 Key Literature

🛣️ Evolution of Self-Improving Agents

Timeline of representative self-improving agent systems, organized by foundation model improvement and scaffolding improvement.

🔖 Papers List

🟦 Foundation Model Improvement

1.1 Intrinsic Generative Demonstrations

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Self-Instruct: Aligning Language Models with Self-Generated Instructions	ACL	paper	code
2023	Large Language Models Can Self-Improve	EMNLP	paper	N/A
2023	Orca: Progressive Learning from Complex Explanation Traces of GPT-4	arXiv	paper	code
2024	SELF: Self-Evolution with Language Feedback	arXiv	paper	N/A
2024	SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning	COLM	paper	code
2025	Improving Model Alignment Through Collective Intelligence of Open-Source LLMS	ICML	paper	N/A
2025	Superficial Self-Improved Reasoners Benefit from Model Merging	EMNLP	paper	code
2025	Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition	arXiv	paper	code
2025	TaskCraft: Automated Generation of Agentic Tasks	arXiv	paper	code
2025	Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning	NeurIPS	paper	code
2025	Maximizing Confidence Alone Improves Reasoning	arXiv	paper	code
2025	DIVE: Diversified Iterative Self-Improvement	arXiv	paper	code
2025	Self-Adapting Language Models	NeurIPS	paper	code
2025	First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training	NeurIPS	paper	code
2025	LADDER: Self-Improving LLMs Through Recursive Problem Decomposition	arXiv	paper	N/A
2025	Self-Consistency Preference Optimization	ICML	paper	N/A
2025	Adapting While Learning: Grounding LLMs for Scientific Problems with Tool Usage Adaptation	ICML	paper	code
2026	Reinforcing General Reasoning Without Verifiers	ICLR	paper	code
2026	SAGE: Multi-Agent Self-Evolution for LLM Reasoning	arXiv	paper	N/A
2026	ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment	arXiv	paper	code
2026	EvoGround: Self-Evolving Video Agents for Video Temporal Grounding	arXiv	paper	code

1.2 Intrinsic Evaluative Feedback

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2022	Constitutional AI: Harmlessness from AI Feedback	arXiv	paper	code
2023	ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent	arXiv	paper	N/A
2025	STRIVE: Structured Reasoning for Self-Improvement in Claim Verification	MIR	paper	N/A
2025	Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models	arXiv	paper	N/A
2025	Self-Evolved Reward Learning for LLMs	ICLR	paper	code
2025	Sample, Predict, then Proceed: Self-Verification Sampling for Tool Use of LLMs	arXiv	paper	N/A
2025	RLSR: Reinforcement Learning from Self Reward	arXiv	paper	N/A
2025	Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization	NeurIPS	paper	code
2025	TTRL: Test-Time Reinforcement Learning	NeurIPS	paper	code
2025	Can Large Reasoning Models Self-Train?	arXiv	paper	code
2025	Self Rewarding Self Improving	arXiv	paper	N/A
2025	Self-Evolving Curriculum for LLM Reasoning	arXiv	paper	code
2025	Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning	arXiv	paper	N/A
2025	Adaptive Self-improvement LLM Agentic System for ML Library Development	ICML	paper	code
2026	Learning to Reason without External Rewards	ICLR	paper	code
2026	Structured Reasoning for Large Language Models	arXiv	paper	N/A
2026	iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models	arXiv	paper	code
2026	STRIVE: Structured Reasoning for Self-improvement in Claim Verification	machine intelligence research	paper	N/A
2026	UniCorn: Towards Self-Improving Unified Multimodal Models through Intrinsic Evaluative Feedback	arXiv	paper	code
2026	Retrospective Progress-Aware Self-Refinement for LLM Agent Training	arXiv	paper	N/A
2026	EVE-Agent: Evidence-Verifiable Self-Evolving Agents	arXiv	paper	N/A

1.3 Extrinsic Exploratory Experience

1.3.1 Interaction with Grounded Task Environments

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation	TMLR	paper	code
2025	Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning	arXiv	paper	code
2025	CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis	COLM	paper	code
2025	LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilities	arXiv	paper	N/A
2025	Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards	arXiv	paper	N/A
2025	WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning	ICLR	paper	code
2025	Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI	ICML	paper	code
2025	DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments	arXiv	paper	code
2025	Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learning	arXiv	paper	N/A
2025	UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents	NeurIPS	paper	code
2025	RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning	arXiv	paper	code
2025	SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience	arXiv	paper	code
2026	WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks	arXiv	paper	code
2026	Kevin: Multi-Turn RL for Generating CUDA Kernels	ICLR	paper	N/A
2026	Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data	arXiv	paper	code
2026	Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills	arXiv	paper	N/A
2026	Self-evolving LLM Agents with In-Distribution Optimization	ICML	paper	N/A
2026	Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills	arXiv	paper	code

1.3.2 Interaction with Simulated Proxy Environments

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Language Models Meet World Models: Embodied Experiences Enhance Language Models	NeurIPS	paper	code
2024	Agent Planning with World Knowledge Model	NeurIPS	paper	code
2025	Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation	ICLR	paper	code
2025	Understanding World or Predicting Future? A Comprehensive Survey of World Models	ACM CSUR	paper	code
2025	General agents contain world models	ICML	paper	N/A
2025	WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model	EMNLP	paper	code
2025	WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis	arXiv	paper	code
2025	GAWM: Global-Aware World Model for Multi-Agent Reinforcement Learning	arXiv	paper	N/A
2025	WMPO: World Model-based Policy Optimization for Vision-Language-Action Models	arXiv	paper	code
2025	Internalizing World Models via Self-Play Finetuning for Agentic RL	arXiv	paper	N/A
2026	AlignUSER: Human-Aligned LLM Agents via World Models for Recommender System Evaluation	arXiv	paper	N/A
2026	WorldEvolver: Self-Evolving World Models for LLM Agent Planning	arXiv	paper	N/A
2025	NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments	ICCV	paper	code
2025	EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory	arXiv	paper	N/A
2026	RISE: Self-Improving Robot Policy with Compositional World Model	RSS	paper	code

🟩 Scaffolding Improvement

2.1 Prompt Optimization

2.1.1 Scalar-Feedback Optimization

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2022	Large Language Models Are Human-Level Prompt Engineers	arXiv	paper	code
2024	Large Language Models as Optimizers	ICLR	paper	code
2024	Prompt Refinement with Image Pivot for Text-to-Image Generation	ACL	paper	code
2024	Learning from Contrastive Prompts: Automated Optimization and Adaptation	arXiv	paper	N/A
2024	PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling	EMNLP	paper	code
2025	The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation	arXiv	paper	N/A
2025	DRO-InstructZero: Distributionally Robust Prompt Optimization for Large Language Models	arXiv	paper	N/A
2025	CoolPrompt: Automatic Prompt Optimization Framework for Large Language Models	FRUCT	paper	N/A
2026	SePO: Self-Evolving Prompt Agent for System Prompt Optimization	arXiv	paper	code
2026	SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration	arXiv	paper	N/A

2.1.2 Qualitative-Feedback Refinement

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Self-Refine: Iterative Refinement with Self-Feedback	arXiv	paper	code
2023	Chain of Hindsight Aligns Language Models with Feedback	arXiv	paper	code
2023	Reflexion: Language Agents with Verbal Reinforcement Learning	NeurIPS	paper	code
2024	Self-Improving Customer Review Response Generation Based on LLMs	COLING	paper	N/A
2024	Prompt Optimization with Human Feedback	ICML Workshop	paper	code
2024	Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs	EMNLP	paper	code
2025	CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation	AAAI	paper	code
2025	Boosting Private Domain Understanding of Efficient MLLMs: A Tuning-free, Adaptive, Universal Prompt Optimization Framework	arXiv	paper	N/A
2026	GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning	ICLR	paper	code
2026	FORGE: Self-Evolving Agent Memory With No Weight Updates via Population Broadcast	CAIS	paper	N/A

2.1.3 Population-Based Evolution

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution	arXiv	paper	N/A
2024	EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers	ICLR	paper	code
2024	WizardLM: Empowering large pre-trained language models to follow complex instructions	ICLR	paper	code
2025	Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings	KDD	paper	N/A
2025	DelvePO: Direction-Guided Self-Evolving Framework for Flexible Prompt Optimization	arXiv	paper	code
2025	How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation	NeurIPS	paper	code

2.1.4 Textual Gradient Optimization

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Automatic Prompt Optimization with "Gradient Descent" and Beam Search	EMNLP	paper	code
2024	How to Correctly do Semantic Backpropagation on Language-based Agentic Systems	arXiv	paper	code
2024	Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs	NeurIPS	paper	code
2025	TextGrad: Automatic "Differentiation" via Text	Nature	paper	code
2025	metaTextGrad: Automatically optimizing language model optimizers	NeurIPS	paper	code
2025	MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimization	NeurIPS Workshop	paper	N/A
2025	Scaling Textual Gradients via Sampling-Based Momentum	ICML	paper	N/A
2026	Pick Your Textual Gradients	arXiv	paper	N/A
2026	SkillOpt: Executive Strategy for Self-Evolving Agent Skills	arXiv	paper	code
2026	VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents	arXiv	paper	N/A
2026	Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems	ACL Findings	paper	N/A
2026	Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies	arXiv	paper	code

2.2 Memory

2.2.1 Memory Object

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Learning to Reason and Memorize with Self-Notes	NeurIPS	paper	N/A
2024	ExpeL: LLM Agents Are Experiential Learners	AAAI	paper	code
2024	A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts	ICML	paper	N/A
2024	CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges	ACL	paper	N/A
2024	MEMORYLLM: Towards Self-Updatable Large Language Models	ICML	paper	code
2025	Agent Workflow Memory	ICML	paper	code
2025	ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory	arXiv	paper	N/A
2025	Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory	arXiv	paper	code
2025	Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory	arXiv	paper	code
2025	PRIME: Planning and Retrieval-Integrated Memory for Enhanced Reasoning	AAAI	paper	N/A
2025	Contextual Memory Reweaving in Large Language Models Using Layered Latent State Reconstruction	arXiv	paper	code
2025	MemGen: Weaving Generative Latent Memory for Self-Evolving Agents	arXiv	paper	code
2025	M+: Extending MemoryLLM with Scalable Long-Term Memory	arXiv	paper	code
2026	Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory	ICLR	paper	code
2026	Thought-Retriever: Don't Just Retrieve Raw Data, Retrieve Thoughts for Memory-Augmented Agentic Systems	arXiv	paper	N/A
2026	Better with Experience: Self-Evolving LLM Agents for Evidence-Grounded Health Community Notes	arXiv	paper	N/A
2026	Agon: An Autonomous Large-Scale Omnidisciplinary Research System Built on Prompt Economy	arXiv	paper	code

2.2.2 Memory Structure

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2022	XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model	ECCV	paper	code
2023	Generative Agents: Interactive Simulacra of Human Behavior	UIST'23	paper	N/A
2024	MemoryBank: Enhancing Large Language Models with Long-Term Memory	arXiv	paper	code
2024	MovieChat: From Dense Token to Sparse Memory for Long Video Understanding	CVPR	paper	code
2024	Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation	ACM MobiCom	paper	code
2025	SCM: Enhancing Large Language Model with Self-Controlled Memory Framework	DASFAA	paper	code
2025	Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents	arXiv	paper	N/A
2025	SALM: A Multi-Agent Framework for Language Model-Driven Social Network Simulation	arXiv	paper	N/A
2025	Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory	arXiv	paper	code
2025	G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems	NeurIPS	paper	code
2025	Zep: A Temporal Knowledge Graph Architecture for Agent Memory	arXiv	paper	N/A
2025	SGMem: Sentence Graph Memory for Long-Term Conversational Agents	arXiv	paper	N/A
2025	CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation	ACL	paper	code
2025	GraphVideoAgent: Enhancing Long-form Video Understanding with Entity Relation Graphs	MM'25	paper	N/A
2025	Decentralizing AI Memory: SHIMI, a Semantic Hierarchical Memory Index for Scalable Agent Reasoning	arXiv	paper	N/A
2025	From Knowledge to Noise: CTIM-Rover and the Pitfalls of Episodic Memory in Software Engineering Agents	ACL Workshop	paper	code
2025	In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents	ACL	paper	N/A
2025	MrSteve: Instruction-Following Agents in Minecraft with What-Where-When Memory	ICLR	paper	code
2026	EvolveMem: Self-Evolving Memory Architecture via AutoResearch for LLM Agents	arXiv	paper	code
2026	SAGE: A Self-Evolving Agentic Graph-Memory Engine for Structure-Aware Associative Memory	arXiv	paper	N/A
2026	Prism: An Evolutionary Memory Substrate for Multi-Agent Open-Ended Discovery	arXiv	paper	N/A
2026	DecentMem: Self-Evolving Multi-Agent Systems via Decentralized Memory	arXiv	paper	N/A
2026	EXG: Self-Evolving Agents with Experience Graphs	arXiv	paper	N/A
2026	CLAG: Adaptive Memory Organization via Agent-Driven Clustering for Small Language Model Agents	ACL Findings	paper	code

2.2.3 Memory Processing

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Generative Agents: Interactive Simulacra of Human Behavior	UIST	paper	code
2024	WizardLM: Empowering large pre-trained language models to follow complex instructions	ICLR	paper	N/A
2025	SEDM: Scalable Self-Evolving Distributed Memory for Agents	ICLR	paper	code
2025	MemInsight: Autonomous Memory Augmentation for LLM Agents	arXiv	paper	N/A
2025	MemGen: Weaving Generative Latent Memory for Self-Evolving Agents	arXiv	paper	code
2025	A-MEM: Agentic Memory for LLM Agents	NeurIPS	paper	code
2025	G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems	arXiv	paper	code
2025	Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory	arXiv	paper	code
2025	Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG	arXiv	paper	code
2025	Memory OS of AI Agent	EMNLP	paper	code
2025	SCM: Enhancing Large Language Model with Self-Controlled Memory Framework	DASFAA	paper	code
2025	Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory	arXiv	paper	code
2025	MLC-Agent: Cognitive Model based on Memory-Learning Collaboration in LLM Empowered Agent Simulation Environment	arXiv	paper	N/A
2026	ForeDreamer: A Self-Evolving Dual-Agent Memory Architecture for Future Event Prediction	EMNLP Findings	paper	code
2025	MemInsight: Autonomous Memory Augmentation for LLM Agents	arXiv	paper	N/A
2026	Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models	ICLR	paper	code
2026	MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory	arXiv	paper	code
2026	TMEM: Scaling Self-Evolving Agents via Parametric Memory	arXiv	paper	N/A
2026	MemQ: Integrating Q-Learning into Self-Evolving Memory for LLM Agents	arXiv	paper	code
2026	Memory Beyond Recall: A Dual-Process Cognitive Memory System for Self-Evolving Agents	arXiv	paper	N/A
2026	AEL: Agent Evolving Learning for Open-Ended Environments	arXiv	paper	code
2026	Metis: Bridging Text and Code Memory for Self-Evolving Agents	arXiv	paper	N/A
2026	Mem^2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation	arXiv	paper	code

2.3 Tool

2.3.1 Dynamic Tool Routing

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	paper	code
2024	ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph	arXiv	paper	N/A
2025	AgentOrchestra: Orchestrating Hierarchical Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol	arXiv	paper	N/A
2025	MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning	arXiv	paper	code
2025	OrchDAG: Complex Tool Orchestration in Multi-Turn Interactions with Plan DAGs	NeurIPS Workshop	paper	N/A
2025	AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learning	arXiv	paper	code
2025	MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Tools	arXiv	paper	code
2025	In-the-Flow Agentic System Optimization for Effective Planning and Tool Use	NeurIPS	paper	code
2025	MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models	arXiv	paper	code
2025	AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol	arXiv	paper	N/A
2025	Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning	NeurIPS	paper	code
2025	Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning	arXiv	paper	code
2025	MCP-Zero: Active Tool Discovery for Autonomous LLM Agents	arXiv	paper	code
2025	AskToAct: Enhancing LLMs Tool Use via Self-Correcting Clarification	arXiv	paper	N/A
2025	MemTool: Optimizing Short-Term Memory Management for Dynamic Tool Calling in LLM Agent Multi-Turn Conversations	arXiv	paper	N/A
2025	Tool-Planner: Task Planning with Clusters across Multiple Tools	ICLR	paper	code
2025	Tool-to-Agent Retrieval: Bridging Tools and Agents for Scalable LLM Multi-Agent Systems	arXiv	paper	N/A
2025	ToolGen: Unified Tool Retrieval and Calling via Generation	ICLR	paper	code
2026	ToolACE-R: Model-aware Iterative Training and Adaptive Refinement for Tool Learning	AAAI	paper	code
2026	DeepAgent: A General Reasoning Agent with Scalable Toolsets	WWW	paper	code
2026	DeepEyesV2: Toward Agentic Multimodal Model	ICLR	paper	code
2026	In-the-Flow Agentic System Optimization for Effective Planning and Tool Use	ICLR	paper	code
2026	GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization	arXiv	paper	code
2026	ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment	arXiv	paper	code

2.3.2 Iterative Tool Refinement

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	paper	code
2025	STELLA: Self-Evolving LLM Agent for Biomedical Research	arXiv	paper	code
2025	SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills	arXiv	paper	code
2025	PyVision: Agentic Vision with Dynamic Tooling	arXiv	paper	code
2025	From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions	ICLR	paper	code
2025	LLMLOOP: Improving LLM-Generated Code and Tests Through Automated Iterative Feedback Loops	ICSME	paper	code
2025	Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis	arXiv	paper	N/A
2026	RewardHarness: Self-Evolving Agentic Post-Training	arXiv	paper	code
2026	MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation	arXiv	paper	N/A
2026	CODESKILL: Learning Self-Evolving Skills for Coding Agents	arXiv	paper	N/A
2026	PFAgent: A Tractable and Self-Evolving Power-Flow Agent for Interactive Grid Analysis	arXiv	paper	N/A

2.3.3 Autonomous Tool Creation

View

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Voyager: An Open-Ended Embodied Agent with Large Language Models	arXiv	paper	code
2024	Large Language Models as Tool Makers	arXiv	paper	code
2024	OS-Copilot: Towards Generalist Computer Agents with Self-Improvement	ICLR	paper	code
2025	Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution	arXiv	paper	code
2025	Alita-G: Self-Evolving Generative Agent for Agent Generation	arXiv	paper	N/A
2025	LLM Agents Making Agent Tools	ACL	paper	code
2025	Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLM	SOSE	paper	N/A
2025	Code2MCP: Transforming Code Repositories into MCP Services	arXiv	paper	code
2025	STELLA: Self-Evolving LLM Agent for Biomedical Research	arXiv	paper	code
2025	PyVision: Agentic Vision with Dynamic Tooling	arXiv	paper	code
2025	AgentOrchestra: Orchestrating Multi-Agent Intelligence with the Tool-Environment-Agent(TEA) Protocol	arXiv	paper	N/A
2025	Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub	ACL	paper	code
2026	OpenSkill: Open-World Self-Evolution for LLM Agents	arXiv	paper	code
2026	EvoDS: Self-Evolving Autonomous Data Science Agent with Skill Learning and Context Management	arXiv	paper	code
2026	Autonomous Evolution of EDA Tools: Multi-Agent Self-Evolved ABC	DAC	paper	N/A
2026	CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification	arXiv	paper	code

2.4 Full Scaffolding

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	Language Agents as Optimizable Graphs	ICML	paper	code
2024	Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation	COLM	paper	code
2024	Automated Design of Agentic Systems	NeurIPS	paper	code
2024	Symbolic Learning Enables Self-Evolving Agents	arXiv	paper	code
2025	Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents	arXiv	paper	code
2025	Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine	arXiv	paper	code
2025	Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement	ACL	paper	code
2025	AlphaEvolve: A coding agent for scientific and algorithmic discovery	arXiv	paper	N/A
2025	ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution	arXiv	paper	code
2025	Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?	arXiv	paper	code
2026	AgentDevel: Reframing Self-Evolving LLM Agents as Release Engineering	arXiv	paper	N/A
2026	JudgeFlow: Agentic Workflow Optimization via Block Judge	arXiv	paper	N/A
2026	RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution	arXiv	paper	code
2026	Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing	arXiv	paper	N/A
2026	Hyperagents	arXiv	paper	code
2026	CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery	COLM	paper	code
2026	Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams	arXiv	paper	code
2026	MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems	arXiv	paper	code
2026	Recursive Self-Evolving Agents via Held-Out Selection	arXiv	paper	N/A
2026	Continual Harness: Online Adaptation for Self-Improving Foundation Agents	arXiv	paper	code
2026	Argus: A General-Purpose Agentic Reasoning Runtime for Long-Horizon Tasks	arXiv	paper	code
2026	The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators	arXiv	paper	N/A
2026	Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories	arXiv	paper	code
2026	LLM-as-Code: Agentic Programming for Agent Harness	KDD AgenticSE Workshop	paper	code
2026	Ouroboros: A Self-Developing Frontier Coding Agent with Reviewed Core Evolution	arXiv	paper	code
2026	Harness Handbook: Making Evolving Agent Harnesses Readable, Navigable, and Editable	arXiv	paper	code

📊 Evaluation & Benchmarking

A representative paper–benchmark matrix for self-improving agents, illustrating which benchmarks are used across different methods. See the Evaluation section of our survey for further details.

The literature is organized along two complementary perspectives:

Measuring Improvement studies how agent progress is assessed, either through explicit metrics or learned/agentic judges.
Benchmarking Improvement groups benchmarks by the improvement mechanism they probe or by the application domain they cover.

Note

A benchmark may appear in more than one subsection because the taxonomy is multi-perspective. For example, a benchmark can provide metric-based measurement while also targeting a specific domain or improvement mechanism.

📏 Measuring Improvement

Metric-Based Measurement

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Mind2Web: Towards a Generalist Agent for the Web	NeurIPS	paper	code
2023	ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills	ICLR	paper	code
2024	CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark	TMLR	paper	code
2024	SWE-Bench+: Enhanced Coding Benchmark for LLMs	arXiv	paper	N/A
2024	WebLINX: Real-World Website Navigation with Multi-Turn Dialogue	ICML	paper	code
2024	GAIA: A Benchmark for General AI Assistants	ICLR	paper	N/A
2024	MINT: Evaluating LLMs in Multi-Turn Interaction with Tools and Language Feedback	ICLR	paper	code
2024	WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?	ICML	paper	code
2025	AgentGym: Evolving Large Language Model-Based Agents across Diverse Environments	ACL	paper	code
2025	GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging	arXiv	paper	code
2025	SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents	arXiv	paper	code
2026	DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agents	WWW	paper	N/A
2026	ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents	ICLR	paper	code
2026	ClawBench: A Benchmark for Evaluating AI Agents on Real-World Online Tasks	arXiv	paper	code

Judge-Based Measurement

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2025	Agent-as-a-Judge: Evaluate Agents with Agents	ICML	paper	code
2025	Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Models	ACL	paper	code
2025	EvalAgent: Discovering Implicit Evaluation Criteria from the Web	COLM	paper	code
2025	Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework (ARJudge)	ACL Findings	paper	N/A
2025	VerifiAgent: A Unified Verification Agent in Language Model Reasoning	EMNLP Findings	paper	code

🧪 Benchmarking Improvement

Mechanism Benchmarks

Foundation-Model-Level

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	SWE-Bench+: Enhanced Coding Benchmark for LLMs	arXiv	paper	N/A
2024	Identifying the Risks of LM Agents with an LM-Emulated Sandbox	ICLR	paper	code
2025	GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging	arXiv	paper	code

Scaffold-Level

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	MINT: Evaluating LLMs in Multi-Turn Interaction with Tools and Language Feedback	ICLR	paper	code
2024	TaskBench: Benchmarking Large Language Models for Task Automation	NeurIPS	paper	code
2024	MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use	ICLR	paper	code
2025	The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models	ICML	paper	code
2026	DrunkAgent: Stealthy Memory Corruption in LLM-Powered Recommender Agents	WWW	paper	N/A
2026	RSI-Bench: Multi-Axis Benchmark for Recursive Self-Improvement	GitHub	N/A	code

Domain Benchmarks

Software Engineering

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	SWE-bench: Can Language Models Resolve Real-World GitHub Issues?	ICLR	paper	code
2024	SWE-Bench+: Enhanced Coding Benchmark for LLMs	arXiv	paper	N/A
2024	SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents	NeurIPS	paper	code
2024	TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved?	arXiv	paper	code
2025	LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering	arXiv	paper	code
2025	DevAI: Automated AI Development Benchmark	ICML	paper	code

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	Mind2Web: Towards a Generalist Agent for the Web	NeurIPS	paper	code
2024	WebArena: A Realistic Web Environment for Building Autonomous Agents	ICLR	paper	code
2024	VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks	ICLR Workshop	paper	code
2024	WebCanvas: Benchmarking Web Agents in Online Environments	ICML Workshop	paper	code
2026	ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents	ICLR	paper	code

Gaming & Strategy

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	clembench: Using Game Play to Evaluate Chat-Optimized Language Models as Conversational Agents	EMNLP	paper	code
2024	clembench-2024: A Challenging, Dynamic, Complementary, Multilingual Benchmark and Underlying Flexible Framework for LLMs as Multi-Action Agents	arXiv	paper	code
2024	GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents	NeurIPS Workshop	paper	code
2024	LLM-Deliberation: Evaluating LLMs with Interactive Multi-Agent Negotiation Game	ICLR Workshop	paper	N/A
2024	GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations	NeurIPS	paper	code

Scientific Discovery

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark	TMLR	paper	code
2024	DiscoveryWorld: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents	NeurIPS Datasets & Benchmarks	paper	code
2025	PaperBench: Evaluating AI's Ability to Replicate AI Research	arXiv	paper	code
2025	PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors	NeurIPS Datasets & Benchmarks	paper	code
2026	AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite	ICLR	paper	code
2026	MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI	arXiv	paper	code

Embodied AI

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2023	ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills	ICLR	paper	code
2025	SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents	arXiv	paper	code
2025	EmbodiedBench: Comprehensive Benchmarking Multi-Modal Large Language Models for Vision-Driven Embodied Agents	ICML	paper	code

General Computer Control

📅 Year	📝 Title	🏛️ Venue	📄 Paper	💻 Code
2024	OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments	NeurIPS	paper	code
2024	AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents	ACL	paper	code
2024	Identifying the Risks of LM Agents with an LM-Emulated Sandbox	ICLR	paper	code
2024	MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use	ICLR	paper	code
2025	Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale	ICML	paper	code
2025	The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models	ICML	paper	code

✍️ Blogs

Metalearning Machines Learn to Learn (1987–) — Jürgen Schmidhuber, 2025 | Blog |
When AI Builds Itself — Anthropic, 2026 | Blog |
AIDE²: The First Evidence of Recursive Self-Improvement — WeCo, 2026 | Blog |
The What & When of Self-Evolving Agents — 2026 | Blog |
Automated Weak-to-Strong Researcher — Anthropic, 2026 | Blog |
The Darwin Gödel Machine: AI That Improves Itself by Rewriting Its Own Code — Sakana AI, 2025 | Blog |
AlphaEvolve: A Gemini-Powered Coding Agent for Designing Advanced Algorithms — Google DeepMind, 2025 | Blog |
FunSearch: Making New Discoveries in Mathematical Sciences Using Large Language Models — Addy Osmani, 2026 | Blog |
Self-Improving Coding Agents — Google DeepMind, 2023 | Blog |

🎙️ Podcasts & Interviews

Jürgen Schmidhuber: Gödel Machines, Meta-Learning, and LSTMs | Lex Fridman Podcast #11 — Lex Fridman Podcast, 2018 | YouTube Video |
再访田渊栋：46.5 亿美金估值的 RSI，与 AI 自进化｜Neolabs 特辑【101 视频播客】 — 2026 | Bilibili Video |
Agent AI Needs Darwin — Machine Learning Street Talk, 2025 | Podcast |
#75 – Marcus Hutter: Universal Artificial Intelligence, AIXI, and AGI — Lex Fridman Podcast, 2020 | Podcast |

🎥 Talks & Courses

Recursive self-improvement (RSI) and meta learning — Jürgen Schmidhuber, 2020 | Video |
2026 北京智源大会｜AI 自进化 — Beijing Academy of Artificial Intelligence, 2026 | Bilibili Video |
Recursive Self-Improvement：当 AI 开始进化 AI — 2026 | Bilibili Video |
全球嘉宾共话 Self-Evolving Agents：从学术前沿到产业落地 — 2026 | Bilibili Video |
【肖仰华复旦教授】大模型驱动的自进化智能体：研究与实践（Agentic AI Summit·2026·深圳站） — 2026 | Bilibili Video |
Jeff Clune | Open-Ended and AI-Generating Algorithms in the Era of Foundation Models — 2025 | YouTube Video |
Stanford CS329A: Self-Improving AI Agents — Stanford University, 2025 | Course | YouTube Playlist |
Open-Ended Agent Learning in the Era of Foundation Models — Jeff Clune | YouTube Video |

🎓 Thesis

Multimodal Agents: From Automation toward Open-Ended Self-Improvement
Mingchen Zhuge, Ph.D. Dissertation, King Abdullah University of Science and Technology (KAUST), 2026.
[Defense page]

🧑‍🏫 Workshop

ICLR 2026 Workshop on AI with Recursive Self-Improvement (RSI 2026)
A workshop dedicated to recursive self-improving AI, held on April 26, 2026, in Rio de Janeiro, Brazil.
Lead organizer: Mingchen Zhuge.

📬 Contact & Community

Discord: Join our server 👉 https://discord.gg/fn5rYJhgaz
WeChat Group: Scan the QR code below to join. If the QR code expires, please add the WeChat admin (13488260597), and we will invite you to the group.

📱 Show WeChat Group QR Code

🤝 Contribute

PRs are welcome!

📌 Citation

@misc{ren2026selfimprovementsmodernagenticsystems,
      title={Self-Improvements in Modern Agentic Systems: A Survey}, 
      author={Zhe Ren and Yimeng Chen and Dandan Guo and Guowei Rong and Tonghui Li and R. B. Xiong and Qingfeng Lan and Wenyi Wang and Li Nanbo and Yibo Yang and Mingchen Zhuge and Jürgen Schmidhuber},
      year={2026},
      eprint={2607.13104},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2607.13104}, 
}