BehR-WM: Behavior Consistency in Text-Based World Models

August 30, 2026 · View on GitHub

Beyond surface-level state consistency: text-based world models should preserve agent behavior, not merely match every token.

BehR-WM method overview

Overview and release scope

BehR-WM studies functional equivalence between a text-based world model and a real environment: does a fixed agent make consistent decisions in the two states? The paper uses this idea for optimization and evaluation.

This public repository intentionally contains only:

serving utilities for released checkpoints;
evaluation code for single-step accuracy, task success and consistency ratios, and step-level behavior consistency;
pinned evaluation-data download tooling.

It does not publish the GRPO loop, training reward implementation, training launcher, or training-data preprocessing. The metadata below records the main reported run settings without representing a runnable training recipe.

Released checkpoints

Environment	Backbone	Hugging Face ID
WebShop	Qwen2.5-7B	`Ricardo-H/BehR-WorldModel-Webshop-Qwen2.5-7B`
WebShop	Llama3.1-8B	`Ricardo-H/BehR-WorldModel-Webshop-Llama3.1-8B`
TextWorld	Qwen2.5-7B	`Ricardo-H/BehR-WorldModel-Textworld-Qwen2.5-7B`
TextWorld	Llama3.1-8B	`Ricardo-H/BehR-WorldModel-Textworld-Llama3.1-8B`

Upstream SFT models

These upstream checkpoints are not ours and are not redistributed here. The full immutable revisions used by the project are:

Environment	Backbone	Upstream model ID	Immutable revision
WebShop	Qwen2.5-7B	`X1AOX1A/WorldModel-Webshop-Qwen2.5-7B`	`99e530d0e6f19c1e61734b61eaec8c2a3a11cabd`
WebShop	Llama3.1-8B	`X1AOX1A/WorldModel-Webshop-Llama3.1-8B`	`1f6d851839519c0acb76c51fd4f9342238046c89`
TextWorld	Qwen2.5-7B	`X1AOX1A/WorldModel-Textworld-Qwen2.5-7B`	`b052a201ae867c3058efba17c9af9cb1635d1f09`
TextWorld	Llama3.1-8B	`X1AOX1A/WorldModel-Textworld-Llama3.1-8B`	`0cc3d6046690ea2f947dca9843531fbcaf7e19c9`

Upstream evaluation data

Evaluation assets are fetched from X1AOX1A/LLMasWorldModels at immutable revision:

ff6ae2b924d1a49e4b89825913887f2ea96cb282

See data/README.md for provenance and per-file details.

Reported training metadata

The following is a concise record of the paper runs, not executable training code or a complete reproduction recipe.

Parameter	Reported value
Algorithm	GRPO
KL coefficient	`0.001`
Batch size	`32`
Maximum prompt length	`14336`
Maximum response length	`1024`
Actor learning rate	`5e-6`
Tensor parallel size	`2`
Group size (`n`)	`5`
Sampling temperature / top-p	`1.3` / `1.0`
Hardware	$4 \times \text{A100}-80\text{GB}$

Repository layout

behr-wm/
├── src/                         # serving clients and agent utilities
├── eval/
│   ├── 01_single_step_accuracy/ # Metric 1
│   ├── 02_task_success_rate/    # Metric 2 and CR/CR_pw
│   └── 03_behavior_consistency/ # standalone Metric 3
├── scripts/
│   ├── env_setup/               # evaluation environment installer
│   ├── servers/                 # checkpoint and environment servers
│   ├── check_data.sh
│   └── download_data.py         # pinned test-asset downloader
├── configs/eval_config.yaml
├── data/init_contexts/          # vendored test init contexts
├── docs/                        # installation and evaluation guides
└── compute_cr.py

Quick start

1. Install

Requirements: Linux, Python 3.10, a compatible CUDA installation, and sufficient GPU memory for the selected checkpoints.

git clone https://github.com/Ricardo-H/behr-wm.git
cd behr-wm
bash scripts/env_setup/install_env.sh
source .venv/bin/activate

See docs/INSTALL.md for details.

2. Download evaluation assets

python scripts/download_data.py
python scripts/download_data.py --env webshop
python scripts/download_data.py --env webshop --webshop-backend
bash scripts/check_data.sh

3. Start services

bash scripts/servers/start_wm_server.sh \
  -m Ricardo-H/BehR-WorldModel-Webshop-Llama3.1-8B -p 8001 -gpu 0
bash scripts/servers/start_webshop_env.sh 36001
bash scripts/servers/start_reference_agent_server.sh \
  -m Qwen/Qwen3-8B -p 8000 -gpu 1

4. Evaluate

# Metric 1: single-step exact match
bash eval/01_single_step_accuracy/run.sh webshop \
  Ricardo-H/BehR-WorldModel-Webshop-Llama3.1-8B outputs/

# Metric 2: task success and consistency ratio
bash eval/02_task_success_rate/run_wm.sh
bash eval/02_task_success_rate/run_wm2real.sh \
  outputs/task_success_rate/wm/webshop/<exp>
bash eval/02_task_success_rate/run_real.sh
python eval/02_task_success_rate/analyze_pairwise_cr.py \
  --real-dir outputs/task_success_rate/real/webshop/<exp> \
  --w2r-dir outputs/task_success_rate/w2r/webshop/<exp>

# Metric 3: step-level behavior consistency
bash eval/03_behavior_consistency/run_eval_bf.sh \
  data/llama_factory/webshop_test_109.json

The W2R argument is the specific <exp> directory produced by the WM rollout, not its parent environment directory. See docs/EVALUATION.md for supported launchers and output schemas. In particular, this release does not claim a real-environment TextWorld launcher for every Metric 2 variant.

Evaluation metrics

Metric 1 — Single-step EM: predicted next-state exact match.
Metric 2 — Task success: success in the world model, action replay in the real environment, real-environment baseline, aggregate CR, and pairwise CR.
Metric 3 — Behavior consistency: compares the fixed agent's logged-action likelihood under a predicted state and its corresponding real state. The release includes this standalone metric implementation; it is not a published training objective implementation.

The task-level consistency ratios are:

$CR = \frac{SR_{W2R}}{SR_{Real}}, \qquad CR_{pw} = \frac{|Real_{\checkmark} \cap W2R_{\checkmark}|}{|Real_{\checkmark}|}$

Documentation

Citation

@article{huang2026behrwm,
  title   = {Beyond State Consistency: Behavior Consistency in Text-Based World Models},
  author  = {Huang, Youling and Chen, Guanqiao and Yao, Junchi and Wang, Lu and
             Yang, Fangkai and Du, Chao and Zhao, ChenZhuo and Zhao, Pu and
             Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei},
  journal = {arXiv preprint arXiv:2604.13824},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.13824}
}

License and acknowledgments

Code is released under Apache-2.0; see LICENSE. Third-party models, data, WebShop, TextWorld, and AgentGym assets retain their upstream licenses and are not redistributed by this repository.

BehR-WM builds on vLLM, WebShop, TextWorld, AgentGym, and the models and splits from From Word to World (Li et al., arXiv:2512.18832).