Awesome Text World Models for LLM-based Agents

June 9, 2026 · View on GitHub

Awesome License: MIT arXiv PRs Welcome

A curated list of papers on Text World Models (TWMs) for LLM-based agents — transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply, thereby supporting planning, efficient learning, and principled evaluation.

This list accompanies our survey Bridging the Agent-World Gap: Text World Models for LLM-based Agents and is organized around the agent lifecycle: Building → Training-Time → Inference-Time → Evaluation.

Timeline of text world model research

🤗 Contributions are welcome! If a paper or repository is missing, incorrect, or has been updated, please open an issue or pull request.

📫 Contact us via emails: liyixia@me.com

📃Please cite our paper if you find our survey or repository helpful!

@misc{li2026textworldmodels,
      title={Bridging the Agent-World Gap: Text World Models for LLM-based Agents},
      author={Yixia Li and Hongru Wang and Peng Lai and Zhiwen Ruan and He Zhu and Youxin Zhu and Ganlong Zhao and Minda Hu and Yun Chen and Sibei Yang and Peng Li and Jeff Z. Pan and Jia Pan and Guanhua Chen and Yang Liu and Guanbin Li},
      year={2026},
      eprint={2606.09032},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.09032},
}

📰 News

🗺️ Overview

The text world model lifecycle. A world model M\mathcal{M} is first constructed via learning, prompting, or code generation; then used to train agents through synthetic rollouts and guide them via lookahead at inference time; and finally evaluated for fidelity and utility.

Text World Model lifecycle overview

📑 Table of Contents

Taxonomy of text world model research

🏗️ 3. Building Text World Models

How a text world model is constructed: by learning (fine-tuning an LLM into a WM), by prompting (eliciting dynamics from a frozen LLM), or by programmatic synthesis (generating executable code as the WM).

Three paradigms for building text world models

3.1 Learning-Based Construction

Fine-tuning a base LLM into a world model via supervised learning on ⟨s,a,s′⟩\langle s, a, s' \rangle trajectories or reinforcement learning.

TitleYearVenuePaperCode
Making Large Language Models into World Models with Precondition and Effect Knowledge2025COLING 2025Paper-
From Word to World: Can Large Language Models be Implicit Text-based World Models?2025arXivPaperCode
CWM: An Open-Weights LLM for Research on Code Generation with World Models2025arXivPaperCode
Agent Learning via Early Experience2025arXivPaper-
RLVR-World: Training World Models with Reinforcement Learning2025arXivPaperCode
WebWorld: A Large-Scale World Model for Web Agent Training2026arXivPaper-
Reinforcement World Model Learning for LLM-based Agents2026arXivPaper-
Self-Improving World Modelling with Latent Actions (SWIRL)2026arXivPaperCode
Beyond State Consistency: Behavior Consistency in Text-Based World Models2026arXivPaperCode
Computer-Using World Model2026arXivPaper-

3.2 Prompt-Based Construction

Turning a frozen LLM into a world model through in-context exemplars, retrieval, or self-evolving memory — without parameter updates.

TitleYearVenuePaperCode
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (LLM-MCTS)2023NeurIPS 2023PaperCode
TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decision2024SIGIR 2024PaperCode
Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents (WebDreamer)2025TMLRPaperCode
Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory2025arXivPaper-
R-WoM: Retrieval-augmented World Model For Computer-use Agents2026arXivPaper-
Aligning Agentic World Models via Knowledgeable Experience Learning (WorldMind)2026arXivPaperCode
Steve-Evolving: Open-World Embodied Self-Evolution via Fine-Grained Diagnosis and Dual-Track Knowledge Distillation2026arXivPaperCode
Test-Time Adaptation for LLM Agents via Environment Interaction2026ICLR 2026PaperCode

3.3 Programmatic Construction (Code as World Model)

Prompting an LLM to emit PDDL, Python, HTML, or DSL programs that an executor runs as the world model; and scaling up environment synthesis.

TitleYearVenuePaperCode
Code World Models for General Game Playing2025arXivPaper-
Towards General Agentic Intelligence via Environment Scaling (AgentScaler)2025arXivPaper-
Web World Models2025arXivPaperCode
Code2World: A GUI World Model via Renderable Code Generation2026arXivPaperCode
SWE-World: Building Software Engineering Agents in Docker-Free Environments2026arXivPaperCode
AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines2026arXivPaperCode
CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion2026arXivPaperCode
Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning2026arXivPaperCode
EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis2026arXivPaperCode
ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training2026arXivPaper-
daVinci-Env: Open SWE Environment Synthesis at Scale2026arXivPaperCode

🎓 4. Training-Time World Models

How world models support agents at training time: by internalizing dynamics into agent parameters, by serving as training environments, or by simulating users.

Three training-time world-model paradigms

4.1 Internalizing World Models into Agent Parameters

Folding world-model predictions into the agent's own weights, as a warm-start or within the reasoning trace.

TitleYearVenuePaperCode
Internalizing World Models via Self-Play Finetuning for Agentic RL (SPA)2025arXivPaperCode
Agent Learning via Early Experience2025arXivPaper-
Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents2025arXivPaper-
Reinforcement World Model Learning for LLM-based Agents2026arXivPaper-
Dyna-Mind: Learning to Simulate from Experience for Better AI Agents2026ICLR 2026Paper-

4.2 World Models as Training Environments

Using a world model to synthesize trajectories offline, serve as an online RL environment, or co-evolve with the agent.

TitleYearVenuePaperCode
WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis2025arXivPaper-
Simulating Environments with Reasoning Models for Agent Training (Simia)2025arXivPaper-
Towards General Agentic Intelligence via Environment Scaling (AgentScaler)2025arXivPaper-
SPICE: Self-Play In Corpus Environments Improves Reasoning2025arXivPaper-
WebEvolver: Enhancing Web Agent Self-Improvement with Co-evolving World Model2025EMNLP 2025PaperCode
Scaling Agent Learning via Experience Synthesis (DreamGym)2026ICLR 2026Paper-
DeepAgent: A General Reasoning Agent with Scalable Toolsets2026WWW 2026PaperCode
DynaWeb: Model-Based Reinforcement Learning of Web Agents2026arXivPaper-

4.3 User Simulation for Agent Training

Modeling the human user as a world model to train multi-turn, proactive, and personalized agents.

TitleYearVenuePaperCode
UserRL: Training Interactive User-Centric Agent via Reinforcement Learning2025arXivPaperCode
Training Proactive and Personalized LLM Agents2025arXivPaperCode
Echo-N1: Affective RL Frontier2025arXivPaper-
HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing2026arXivPaperCode
HumanLM: Simulating Users with State Alignment Beats Response Imitation2026arXivPaper-
Flipping the Dialogue: Training and Evaluating User Language Models (UserLM)2026ICLR 2026Paper-
Learning Personalized Agents from Human Feedback (PAHF)2026arXivPaper-
Cold-Start Personalization via Training-Free Priors from Structured World Models (Pep)2026arXivPaper-
OpenClaw-RL: Train Any Agent Simply by Talking2026arXivPaperCode

🔮 5. Inference-Time World Models

How world models guide agents at inference time, as a simulator for lookahead/search or as a verifier of proposed actions.

Inference-time roles of a text world model

5.1 World Model as Simulator: Shallow Lookahead

Imagining the immediate consequence of each candidate action and picking the best.

TitleYearVenuePaperCode
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (WMA)2025ICLR 2025Paper-
Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents (WebDreamer)2025TMLRPaperCode
SimuRA: A World-Model-Driven Simulative Reasoning Architecture for General Goal-Oriented Agents2025arXivPaper-
WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents2025NeurIPS 2025PaperCode

Using the world model as a transition function for multi-step rollouts and search (e.g., MCTS).

TitleYearVenuePaperCode
Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (LLM-MCTS)2023NeurIPS 2023PaperCode
Reasoning with Language Model is Planning with World Model (RAP)2023EMNLP 2023PaperCode
Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models (LATS)2024ICML 2024PaperCode
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents2024arXivPaper-
Improving LLM Agent Planning with In-Context Learning via Atomic Fact Augmentation and Lookahead Search2025ICML 2025 WorkshopPaper-
Code World Models for General Game Playing2025arXivPaper-
Synthesizing World Models for Bilevel Planning (TheoryCoder)2025TMLRPaper-

5.3 World Model as Verifier

The world model predicts the consequence of a proposed action, and a judge accepts it or sends it back for revision.

TitleYearVenuePaperCode
From Word to World: Can Large Language Models be Implicit Text-based World Models?2025arXivPaperCode
SWE-World: Building Software Engineering Agents in Docker-Free Environments2026arXivPaperCode
Computer-Using World Model2026arXivPaper-
Can We Predict Before Executing Machine Learning Agents? (FOREAGENT)2026arXivPaperCode
Budget-Constrained Agentic Large Language Models: Intention-Based Planning for Costly Tool Use (INTENT)2026arXivPaper-
World-Model-Augmented Web Agents with Action Correction (WAC)2026arXivPaper-

📊 6. Evaluation

How text world models are evaluated — both the fidelity of the world model itself and its use as an evaluation environment for agents.

Three evaluation paradigms for text world models

6.1 Evaluating World Models Themselves

Measuring prediction accuracy, consistency, and task-driven utility of the world model.

TitleYearVenuePaperCode
Can Language Models Serve as Text-Based World Simulators? (ByteSized32)2024ACL 2024PaperCode
From Word to World: Can Large Language Models be Implicit Text-based World Models?2025arXivPaperCode
WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning2025ICML 2025 WorkshopPaperCode
Text2World: Benchmarking Large Language Models for Symbolic World Model Generation2025arXivPaperCode
The Safety Challenge of World Models for Embodied AI Agents: A Review2025arXivPaper-
LLM-Based World Models Can Make Decisions Solely, But Rigorous Evaluations are Needed2026TMLRPaperCode
What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm for Studying Environment Understanding2026arXivPaperCode
Beyond State Consistency: Behavior Consistency in Text-Based World Models2026arXivPaperCode

6.2 Benchmark Design (WM as Evaluation Environment)

Using world models to construct benchmarks and interactive evaluation environments for agents.

TitleYearVenuePaperCode
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment2025arXivPaperCode
MobileWorldBench: Towards Semantic World Modeling For Mobile Agents2025arXivPaperCode
AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning2025arXivPaperCode
LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment2025EMNLP 2025Paper-
RECODE-H: A Benchmark for Research Code Development with Interactive Human Feedback2026ICLR 2026PaperCode
LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation2026arXivPaperCode

6.3 Simulator Validity

Studying how faithfully a world-model simulator reflects the real environment or real users.

TitleYearVenuePaperCode
Large Emotional World Model2025arXivPaper-
IDRBench: Interactive Deep Research Benchmark2026arXivPaper-
Interactive Benchmarks2026arXivPaperCode

📝 Citation

If you find this survey and repository useful for your research, please consider citing:

@misc{li2026textworldmodels,
      title={Bridging the Agent-World Gap: Text World Models for LLM-based Agents},
      author={Yixia Li and Hongru Wang and Peng Lai and Zhiwen Ruan and He Zhu and Youxin Zhu and Ganlong Zhao and Minda Hu and Yun Chen and Sibei Yang and Peng Li and Jeff Z. Pan and Jia Pan and Guanhua Chen and Yang Liu and Guanbin Li},
      year={2026},
      eprint={2606.09032},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.09032},
}

🙌 Contributing

This list is maintained alongside the survey. To add or update an entry:

| Title | Year | Venue | Paper | Code |
|-------|------|-------|-------|------|
| **Paper Title** | 2025 | Conference/Journal Name | [Paper](https://arxiv.org/abs/xxxx.xxxxx) | [Code](https://github .com/username/repo) |

Pull requests and issues are warmly welcomed!