LLM Game Agent (ALFWorld Multi-Turn)

Author: Haofeiy

This example demonstrates training a language model to complete household tasks in the ALFWorld text-based environment.

Overview

ALFWorld is a text-based interactive environment for training agents to complete household tasks. Tasks include:

• Pick and place objects
• Look at objects under light
• Clean objects and place them
• Heat/cool objects and place them
• Pick multiple objects

Prerequisites

1. Complete the Installation steps
2. Install ALFWorld: pip install alfworld
3. Download ALFWorld data: alfworld-download
4. Get your IP address: hostname -I

Step 1: Start the Scheduler (Server Side)

bash opentinker/scripts/launch_scheduler.sh --scheduler-port <scheduler_port>

Step 2: Start the ALFWorld Environment (Client Side)

python -m opentinker.environment.alfworld.alfworld_server \
    --port <env_port> \
    --max_steps 50 \
    --split train \
    --num_games 5 # can be larger or smaller

Server Options:

• --port: Server port (default: 8082)
• --max_steps: Max steps per episode (default: 50)
• --split: Dataset split (train, eval_in_distribution, eval_out_of_distribution)
• --num_games: Number of games to load (-1 = all, use smaller value for faster loading)

Step 3: Run Training

python opentinker/client/alfworld_rl.py \
    tokenizer_path=Qwen/Qwen2.5-3B-Instruct \
    batch_size=4 \
    val_batch_size=50 \
    num_steps=1000 \
    save_freq=20000 \
    test_freq=10 \
    scheduler_url=http://<server_endpoint>:<scheduler_port> \
    interaction.config.env_port=<env_port> \
    interaction.config.env_host=<client_endpoint>

Training Parameters:

• num_steps: Total training steps (alternative: use num_epochs)
• batch_size: Training batch size
• val_batch_size: Validation samples per evaluation
• test_freq: Validation frequency (every N steps)
• adv_estimator: Advantage estimator (gae, grpo, grpo_per_step)

Reward Structure

Event	Reward
Task Success	+10.0
Task Failure	-1.0
Per Step Penalty	-0.01
Invalid Action	-0.1

Example Actions

The agent interacts with the environment using text commands:

• go to desk 1 - Navigate to a location
• take book 1 from desk 1 - Pick up an object
• put book 1 in/on shelf 1 - Place an object
• open drawer 1 - Open a container
• use lamp 1 - Use a device
• examine book 1 - Look at an object
• inventory - Check held items
• look - Look around

Configuration Reference

See opentinker/client/client_config/alfworld_param.yaml for full configuration options.