Wan2.1 Fun Full Parameter Training Guide

July 14, 2026 · View on GitHub

This document provides a complete workflow for full parameter training of Wan2.1 Fun Diffusion Transformer, including environment setup, data preparation, distributed training, and inference testing.

Note: Wan2.1 Fun is a video generation model that supports Text-to-Video (T2V), Image-to-Video (I2V), and Video Inpainting. This document covers the training workflow for both normal video generation and video inpainting tasks.

1. Environment Setup
2. Data Preparation
3. Full Parameter Training
4. Inference Testing
5. Additional Resources

1. Environment Setup

Option 1: Using requirements.txt

pip install -r requirements.txt

Option 2: Manual Installation

pip install Pillow einops safetensors timm tomesd librosa "torch>=2.1.2" torchdiffeq torchsde decord datasets numpy scikit-image
pip install omegaconf SentencePiece imageio[ffmpeg] imageio[pyav] tensorboard beautifulsoup4 ftfy func_timeout onnxruntime
pip install "peft>=0.17.0" "accelerate>=0.25.0" "gradio>=3.41.2" "diffusers>=0.30.1" "transformers>=4.46.2"
pip install yunchang xfuser modelscope openpyxl
pip uninstall opencv-python opencv-contrib-python opencv-python-headless -y
pip install opencv-python-headless
pip install deepspeed==0.17.0 numpy==1.26.4

Option 3: Using Docker

When using Docker, ensure that the GPU driver and CUDA environment are properly installed on your machine, then execute the following commands:

# pull image
docker pull mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun

# enter image
docker run -it -p 7860:7860 --network host --gpus all --security-opt seccomp:unconfined --shm-size 200g mybigpai-public-registry.cn-beijing.cr.aliyuncs.com/easycv/torch_cuda:cogvideox_fun

2. Data Preparation

2.1 Quick Test Dataset

We provide a test dataset containing several training samples.

# Download official demo dataset
modelscope download --dataset PAI/X-Fun-Videos-Demo --local_dir ./datasets/X-Fun-Videos-Demo

2.2 Dataset Structure

📦 datasets/
├── 📂 my_dataset/
│   ├── 📂 train/
│   │   ├── 📄 video001.mp4
│   │   ├── 📄 video002.mp4
│   │   └── 📄 ...
│   └── 📄 metadata.json

2.3 metadata.json Format

Relative Path Format (example format):

[
  {
    "file_path": "train/video001.mp4",
    "text": "A beautiful sunset over the ocean, golden hour lighting",
    "type": "video",
    "width": 1024,
    "height": 1024
  },
  {
    "file_path": "train/video002.mp4",
    "text": "A person walking through a forest, cinematic view",
    "type": "video",
    "width": 1328,
    "height": 1328
  }
]

Absolute Path Format:

[
  {
    "file_path": "/mnt/data/videos/sunset.mp4",
    "text": "A beautiful sunset over the ocean",
    "type": "video",
    "width": 1024,
    "height": 1024
  }
]

Key Field Descriptions:

file_path: Video path (relative or absolute path)
text: Video description (English prompt)
type: Data type, should be "video"
width / height: Video width and height (recommended to provide, used for bucket training. If not provided, they will be automatically read during training, which may affect training speed when data is stored on slow storage systems like OSS).
- You can use scripts/process_json_add_width_and_height.py to extract width and height from JSON files without these fields. It supports processing both images and videos.
- Usage: python scripts/process_json_add_width_and_height.py --input_file datasets/X-Fun-Videos-Demo/metadata.json --output_file datasets/X-Fun-Videos-Demo/metadata_add_width_height.json.

2.4 Relative vs Absolute Path Usage

Relative Path:

If your data uses relative paths, configure in the training script:

export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"

Absolute Path:

If your data uses absolute paths, configure in the training script:

export DATASET_NAME=""
export DATASET_META_NAME="/mnt/data/metadata_add_width_height.json"

💡 Recommendation: If the dataset is small and stored locally, use relative paths. If the dataset is stored on external storage (e.g., NAS, OSS) or shared across multiple machines, use absolute paths.

3. Full Parameter Training

3.1 Download Pretrained Model

# Create model directory
mkdir -p models/Diffusion_Transformer

# Download official Wan2.1 Fun weights
# Inpaint model (Video Inpainting)
modelscope download --model PAI/Wan2.1-Fun-V1.1-1.3B-InP --local_dir models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP

# modelscope download --model PAI/Wan2.1-Fun-V1.1-14B-InP --local_dir models/Diffusion_Transformer/Wan2.1-Fun-V1.1-14B-InP

3.2 Quick Start (DeepSpeed-Zero-2)

After downloading the dataset as in 2.1 Quick Test Dataset and the pretrained model as in 3.1 Download Pretrained Model, you can directly copy and run the quick start command.

We recommend using DeepSpeed-Zero-2 or FSDP for training. Here we use DeepSpeed-Zero-2 as an example.

The difference between DeepSpeed-Zero-2 and FSDP in this repository is whether the model weights are sharded. If GPU memory is insufficient when using multiple GPUs with DeepSpeed-Zero-2, you can switch to FSDP for training.

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

accelerate launch --use_deepspeed --deepspeed_config_file config/zero_stage2_config.json --deepspeed_multinode_launcher standard scripts/wan2.1_fun/train.py \
  --config_path="config/wan2.1/wan_civitai.yaml" \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$DATASET_NAME \
  --train_data_meta=$DATASET_META_NAME \
  --image_sample_size=640 \
  --video_sample_size=640 \
  --token_sample_size=640 \
  --video_sample_stride=2 \
  --video_sample_n_frames=81 \
  --train_batch_size=1 \
  --video_repeat=1 \
  --gradient_accumulation_steps=1 \
  --dataloader_num_workers=8 \
  --num_train_epochs=100 \
  --checkpointing_steps=50 \
  --learning_rate=2e-05 \
  --lr_scheduler="constant_with_warmup" \
  --lr_warmup_steps=100 \
  --seed=42 \
  --output_dir="output_dir_wan2.1_fun" \
  --gradient_checkpointing \
  --mixed_precision="bf16" \
  --adam_weight_decay=3e-2 \
  --adam_epsilon=1e-10 \
  --vae_mini_batch=1 \
  --max_grad_norm=0.05 \
  --random_hw_adapt \
  --training_with_video_token_length \
  --enable_bucket \
  --uniform_sampling \
  --low_vram \
  --train_mode="inpaint" \
  --trainable_modules "."

3.3 Training Parameter Reference

Key Parameter Descriptions:

Parameter	Description	Example Value
`--pretrained_model_name_or_path`	Pretrained model path	`models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP`
`--train_data_dir`	Training data directory	`datasets/X-Fun-Videos-Demo/`
`--train_data_meta`	Training data metadata file	`datasets/X-Fun-Videos-Demo/metadata_add_width_height.json`
`--train_batch_size`	Batch size	1
`--image_sample_size`	Maximum training resolution for images	640
`--video_sample_size`	Maximum training resolution for videos	640
`--token_sample_size`	Token sample size	640
`--video_sample_stride`	Video sample stride	2
`--video_sample_n_frames`	Number of video frames to sample	81
`--gradient_accumulation_steps`	Gradient accumulation steps (effectively increases batch size)	1
`--dataloader_num_workers`	Number of DataLoader workers	8
`--num_train_epochs`	Number of training epochs	100
`--checkpointing_steps`	Save checkpoint every N steps	50
`--learning_rate`	Initial learning rate	2e-05
`--lr_scheduler`	Learning rate scheduler	`constant_with_warmup`
`--lr_warmup_steps`	Learning rate warmup steps	100
`--seed`	Random seed	42
`--output_dir`	Output directory	`output_dir_wan2.1_fun`
`--gradient_checkpointing`	Gradient checkpointing	-
`--mixed_precision`	Mixed precision: `fp16/bf16`	`bf16`
`--adam_weight_decay`	AdamW weight decay	3e-2
`--adam_epsilon`	AdamW epsilon value	1e-10
`--vae_mini_batch`	Mini-batch size for VAE encoding	1
`--max_grad_norm`	Maximum gradient norm for clipping	0.05
`--enable_bucket`	Enable bucket training without cropping, groups by resolution	-
`--random_hw_adapt`	Randomly scale images/videos to `[min_size, max_size]`	-
`--training_with_video_token_length`	Train based on token length, supports arbitrary resolutions	-
`--uniform_sampling`	Uniform timestep sampling	-
`--low_vram`	Low VRAM mode	-
`--train_mode`	Training mode: `inpaint` (video inpainting)	`inpaint`
`--resume_from_checkpoint`	Resume training from checkpoint, use `"latest"` to auto-select	None
`--validation_steps`	Run validation every N steps	100
`--validation_epochs`	Run validation every N epochs	500
`--validation_prompts`	Prompts for validation video generation	`"A brown dog shaking its head..."`
`--validation_paths`	Reference image paths for inpaint validation (inpaint mode only)	`"asset/1.png"`
`--trainable_modules`	Trainable modules (`"."` means all modules)	`"."`

Sample Size Configuration Guide:

video_sample_size represents the resolution size for videos; when random_hw_adapt is True, it represents the minimum value for both video and image resolutions.
image_sample_size represents the resolution size for images; when random_hw_adapt is True, it represents the maximum value for both video and image resolutions.
token_sample_size represents the resolution corresponding to the maximum token length when training_with_video_token_length is True.
Due to potential confusion in configuration, if you don't require arbitrary resolution for finetuning, it is recommended to set video_sample_size, image_sample_size, and token_sample_size to the same fixed value, such as (320, 480, 512, 640, 960).
- All set to 320 represents 240P.
- All set to 480 represents 320P.
- All set to 640 represents 480P.
- All set to 960 represents 720P.

Token Length Training Explanation:

When training_with_video_token_length is enabled, the model trains based on token length.
For example: a video with 512x512 resolution and 49 frames has a token length of 13,312, requiring token_sample_size = 512.
- At 512x512 resolution, the number of video frames is 49 (~= 512 * 512 * 49 / 512 / 512).
- At 768x768 resolution, the number of video frames is 21 (~= 512 * 512 * 49 / 768 / 768).
- At 1024x1024 resolution, the number of video frames is 9 (~= 512 * 512 * 49 / 1024 / 1024).
- These resolutions combined with their corresponding frame counts allow the model to generate videos of different sizes.

3.4 Training Validation

You can configure validation parameters to periodically generate test videos during training, allowing you to monitor training progress and model quality.

Validation Parameter Descriptions:

Parameter	Description	Recommended Value
`--validation_steps`	Run validation every N steps	100
`--validation_epochs`	Run validation every N epochs	500
`--validation_prompts`	Prompts for validation video generation	English prompts
`--validation_paths`	Reference image paths for inpaint validation (inpaint mode only)	`"asset/1.png"`

Inpaint Mode Example (Video Inpainting Validation):

  --validation_paths "asset/1.png" \
  --validation_steps=100 \
  --validation_epochs=500 \
  --validation_prompts="A brown dog shaking its head, sitting on a light-colored sofa in a cozy room. Behind the dog, there's a framed painting on a shelf, surrounded by pink flowers. The soft, warm lighting in the room creates a comfortable atmosphere."

Notes:

Validation videos will be saved to the output_dir directory
Multiple prompts format: --validation_prompts "prompt1" "prompt2" "prompt3"
inpaint mode requires the --validation_paths parameter

3.5 Training with FSDP

If GPU memory is insufficient when using multiple GPUs with DeepSpeed-Zero-2, you can switch to FSDP for training.

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

accelerate launch --mixed_precision="bf16" --use_fsdp --fsdp_auto_wrap_policy TRANSFORMER_BASED_WRAP --fsdp_transformer_layer_cls_to_wrap=WanAttentionBlock --fsdp_sharding_strategy "FULL_SHARD" --fsdp_state_dict_type=SHARDED_STATE_DICT --fsdp_backward_prefetch "BACKWARD_PRE" --fsdp_cpu_ram_efficient_loading False scripts/wan2.1_fun/train.py \
  --config_path="config/wan2.1/wan_civitai.yaml" \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$DATASET_NAME \
  --train_data_meta=$DATASET_META_NAME \
  --image_sample_size=640 \
  --video_sample_size=640 \
  --token_sample_size=640 \
  --video_sample_stride=2 \
  --video_sample_n_frames=81 \
  --train_batch_size=1 \
  --video_repeat=1 \
  --gradient_accumulation_steps=1 \
  --dataloader_num_workers=8 \
  --num_train_epochs=100 \
  --checkpointing_steps=50 \
  --learning_rate=2e-05 \
  --lr_scheduler="constant_with_warmup" \
  --lr_warmup_steps=100 \
  --seed=42 \
  --output_dir="output_dir_wan2.1_fun" \
  --gradient_checkpointing \
  --mixed_precision="bf16" \
  --adam_weight_decay=3e-2 \
  --adam_epsilon=1e-10 \
  --vae_mini_batch=1 \
  --max_grad_norm=0.05 \
  --random_hw_adapt \
  --training_with_video_token_length \
  --enable_bucket \
  --uniform_sampling \
  --low_vram \
  --train_mode="inpaint" \
  --trainable_modules "."

3.6 Other Backends

3.6.1 Training with DeepSpeed-Zero-3

DeepSpeed Zero-3 is not highly recommended at the moment. In this repository, using FSDP has fewer errors and is more stable.

DeepSpeed Zero-3 is suitable for 14B Wan at high resolutions. After training, you can use the following command to get the final model:

python scripts/zero_to_bf16.py output_dir/checkpoint-{our-num-steps} output_dir/checkpoint-{your-num-steps}-outputs --max_shard_size 80GB --safe_serialization

Training shell command is as follows:

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

accelerate launch --zero_stage 3 --zero3_save_16bit_model true --zero3_init_flag true --use_deepspeed --deepspeed_config_file config/zero_stage3_config.json --deepspeed_multinode_launcher standard scripts/wan2.1_fun/train.py \
  --config_path="config/wan2.1/wan_civitai.yaml" \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$DATASET_NAME \
  --train_data_meta=$DATASET_META_NAME \
  --image_sample_size=640 \
  --video_sample_size=640 \
  --token_sample_size=640 \
  --video_sample_stride=2 \
  --video_sample_n_frames=81 \
  --train_batch_size=1 \
  --video_repeat=1 \
  --gradient_accumulation_steps=1 \
  --dataloader_num_workers=8 \
  --num_train_epochs=100 \
  --checkpointing_steps=50 \
  --learning_rate=2e-05 \
  --lr_scheduler="constant_with_warmup" \
  --lr_warmup_steps=100 \
  --seed=42 \
  --output_dir="output_dir_wan2.1_fun" \
  --gradient_checkpointing \
  --mixed_precision="bf16" \
  --adam_weight_decay=3e-2 \
  --adam_epsilon=1e-10 \
  --vae_mini_batch=1 \
  --max_grad_norm=0.05 \
  --random_hw_adapt \
  --training_with_video_token_length \
  --enable_bucket \
  --uniform_sampling \
  --low_vram \
  --train_mode="inpaint" \
  --trainable_modules "."

3.6.2 Training Without DeepSpeed and FSDP

This approach is not recommended as it lacks memory-saving backends and may easily cause out-of-memory errors. Provided here for reference only.

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

accelerate launch --mixed_precision="bf16" scripts/wan2.1_fun/train.py \
  --config_path="config/wan2.1/wan_civitai.yaml" \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$DATASET_NAME \
  --train_data_meta=$DATASET_META_NAME \
  --image_sample_size=640 \
  --video_sample_size=640 \
  --token_sample_size=640 \
  --video_sample_stride=2 \
  --video_sample_n_frames=81 \
  --train_batch_size=1 \
  --video_repeat=1 \
  --gradient_accumulation_steps=1 \
  --dataloader_num_workers=8 \
  --num_train_epochs=100 \
  --checkpointing_steps=50 \
  --learning_rate=2e-05 \
  --lr_scheduler="constant_with_warmup" \
  --lr_warmup_steps=100 \
  --seed=42 \
  --output_dir="output_dir_wan2.1_fun" \
  --gradient_checkpointing \
  --mixed_precision="bf16" \
  --adam_weight_decay=3e-2 \
  --adam_epsilon=1e-10 \
  --vae_mini_batch=1 \
  --max_grad_norm=0.05 \
  --random_hw_adapt \
  --training_with_video_token_length \
  --enable_bucket \
  --uniform_sampling \
  --low_vram \
  --train_mode="inpaint" \
  --trainable_modules "."

3.7 Multi-Node Distributed Training

Suitable for: Ultra-large-scale datasets, faster training speed

3.7.1 Environment Configuration

Assuming 2 machines with 8 GPUs each:

Machine 0 (Master):

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
export MASTER_ADDR="192.168.1.100"  # Master machine IP
export MASTER_PORT=10086
export WORLD_SIZE=2                  # Total number of machines
export NUM_PROCESS=16                # Total processes = machines × 8
export RANK=0                        # Current machine rank (0 or 1)
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

accelerate launch --mixed_precision="bf16" --main_process_ip=$MASTER_ADDR --main_process_port=$MASTER_PORT --num_machines=$WORLD_SIZE --num_processes=$NUM_PROCESS --machine_rank=$RANK --use_deepspeed --deepspeed_config_file config/zero_stage2_config.json --deepspeed_multinode_launcher standard scripts/wan2.1_fun/train.py \
  --config_path="config/wan2.1/wan_civitai.yaml" \
  --pretrained_model_name_or_path=$MODEL_NAME \
  --train_data_dir=$DATASET_NAME \
  --train_data_meta=$DATASET_META_NAME \
  --image_sample_size=640 \
  --video_sample_size=640 \
  --token_sample_size=640 \
  --video_sample_stride=2 \
  --video_sample_n_frames=81 \
  --train_batch_size=1 \
  --video_repeat=1 \
  --gradient_accumulation_steps=1 \
  --dataloader_num_workers=8 \
  --num_train_epochs=100 \
  --checkpointing_steps=50 \
  --learning_rate=2e-05 \
  --lr_scheduler="constant_with_warmup" \
  --lr_warmup_steps=100 \
  --seed=42 \
  --output_dir="output_dir_wan2.1_fun" \
  --gradient_checkpointing \
  --mixed_precision="bf16" \
  --adam_weight_decay=3e-2 \
  --adam_epsilon=1e-10 \
  --vae_mini_batch=1 \
  --max_grad_norm=0.05 \
  --random_hw_adapt \
  --training_with_video_token_length \
  --enable_bucket \
  --uniform_sampling \
  --low_vram \
  --train_mode="inpaint" \
  --trainable_modules "."

Machine 1 (Worker):

export MODEL_NAME="models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"
export DATASET_NAME="datasets/X-Fun-Videos-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Demo/metadata_add_width_height.json"
export MASTER_ADDR="192.168.1.100"  # Same as Master
export MASTER_PORT=10086
export WORLD_SIZE=2
export NUM_PROCESS=16
export RANK=1  # Note: this is 1
# NCCL_IB_DISABLE=1 and NCCL_P2P_DISABLE=1 are used in multi nodes without RDMA. 
# export NCCL_IB_DISABLE=1
# export NCCL_P2P_DISABLE=1
NCCL_DEBUG=INFO

# Use the same accelerate launch command as Machine 0

3.7.2 Multi-Node Training Notes

Network Requirements:
- Recommended: RDMA/InfiniBand (high performance)
- Without RDMA, add environment variables:
```
export NCCL_IB_DISABLE=1
export NCCL_P2P_DISABLE=1
```
Data Synchronization: All machines must have access to the same data paths (NFS/shared storage)

4. Inference Testing

4.1 Inference Parameter Reference

Key Parameter Descriptions:

Parameter	Description	Example Value
`GPU_memory_mode`	GPU memory management mode, see table below	`model_group_offload`
`ulysses_degree`	Head dimension parallelism, 1 for single GPU	1
`ring_degree`	Sequence dimension parallelism, 1 for single GPU	1
`fsdp_dit`	Use FSDP for Transformer during multi-GPU inference	`False`
`fsdp_text_encoder`	Use FSDP for text encoder during multi-GPU inference	`True`
`compile_dit`	Compile Transformer for faster inference (effective for fixed resolution)	`False`
`model_name`	Model path	`models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP`
`sampler_name`	Sampler type: `Flow`, `Flow_Unipc`, `Flow_DPM++`	`Flow_Unipc`
`transformer_path`	Path to trained Transformer weights	`None`
`vae_path`	Path to trained VAE weights	`None`
`lora_path`	LoRA weights path	`None`
`sample_size`	Generated video resolution `[height, width]`	`[480, 832]` or `[832, 480]`
`video_length`	Number of video frames	`81`
`fps`	Frames per second	`16`
`weight_dtype`	Model weight precision, use `torch.float16` for GPUs not supporting bf16	`torch.bfloat16`
`validation_image_start`	Reference image path for inpaint mode	`"asset/1.png"`
`validation_image_end`	Target image path for inpaint mode (set to `None` for pure I2V)	`None`
`prompt`	Positive prompt describing content	`"A brown dog shaking its head..."`
`negative_prompt`	Negative prompt to avoid certain content	`"low resolution, low quality..."`
`guidance_scale`	Guidance strength	6.0
`seed`	Random seed for reproducibility	43
`num_inference_steps`	Number of inference steps	50
`lora_weight`	LoRA weight strength	0.55
`save_path`	Path to save generated video	`samples/wan-videos-fun-t2v` or `samples/wan-videos-fun-i2v`

GPU Memory Management Modes:

Mode	Description	Memory Usage
`model_full_load`	Full model loaded to GPU	Highest
`model_full_load_and_qfloat8`	Full load + FP8 quantization	High
`model_cpu_offload`	Offload model to CPU after use	Medium
`model_cpu_offload_and_qfloat8`	CPU offload + FP8 quantization	Medium-Low
`model_group_offload`	Layer groups switch between CPU/CUDA	Low
`sequential_cpu_offload`	Layer-by-layer offload (slowest)	Lowest

4.2 Text-to-Video (T2V) Inference

Run the following command for single GPU inference:

python examples/wan2.1_fun/predict_t2v.py

Edit examples/wan2.1_fun/predict_t2v.py according to your needs. For initial inference, focus on the following parameters. If you're interested in other parameters, refer to the inference parameter reference above.

# Choose based on GPU memory
GPU_memory_mode = "sequential_cpu_offload"
# Based on actual model path
model_name = "models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"  
# Path to trained weights, e.g., "output_dir_wan2.1_fun/checkpoint-xxx/diffusion_pytorch_model.safetensors"
transformer_path = None  
# Write based on the content you want to generate
prompt = "A brown dog shaking its head, sitting on a light-colored sofa in a cozy room. Behind the dog, there's a framed painting on a shelf, surrounded by pink flowers. The soft, warm lighting in the room creates a comfortable atmosphere."  
# ...

4.3 Image-to-Video (I2V) Inference

Run the following command for single GPU inference:

python examples/wan2.1_fun/predict_i2v.py

Edit examples/wan2.1_fun/predict_i2v.py according to your needs. For initial inference, focus on the following parameters. If you're interested in other parameters, refer to the inference parameter reference above.

# Choose based on GPU memory
GPU_memory_mode = "sequential_cpu_offload"
# Based on actual model path
model_name = "models/Diffusion_Transformer/Wan2.1-Fun-V1.1-1.3B-InP"  
# Path to trained weights, e.g., "output_dir_wan2.1_fun/checkpoint-xxx/diffusion_pytorch_model.safetensors"
transformer_path = None  
# Starting image for image-to-video / inpaint
validation_image_start = "asset/1.png"
# Target image for inpaint (set to None for pure image-to-video)
validation_image_end = None
# Write based on the content you want to generate
prompt = "A brown dog shaking its head, sitting on a light-colored sofa in a cozy room. Behind the dog, there's a framed painting on a shelf, surrounded by pink flowers. The soft, warm lighting in the room creates a comfortable atmosphere."  
# ...

4.4 Multi-GPU Parallel Inference

Suitable for: High-resolution generation, accelerated inference

Install Parallel Inference Dependencies

pip install xfuser==0.4.2 yunchang==0.6.2

Configure Parallel Strategy

Edit examples/wan2.1_fun/predict_t2v.py or examples/wan2.1_fun/predict_i2v.py:

``$\text{python}

\text{Ensure} \text{ulysses_degree} \times \text{ring_degree} = \text{number} \text{of} \text{GPUs} \text{used}

\text{For} \text{example}, \text{using} 2 \text{GPUs}:

\text{ulysses_degree} = 2 # \text{Head} \text{dimension} \text{parallelism} \text{ring_degree} = 1 # \text{Sequence} \text{dimension} \text{parallelism} $``

Configuration Principles:

ulysses_degree must be divisible by the model's number of heads
ring_degree splits along the sequence dimension and affects communication overhead; avoid using it when heads can be evenly divided

Configuration Examples:

GPU Count	ulysses_degree	ring_degree	Description
1	1	1	Single GPU
4	4	1	Head parallelism
8	8	1	Head parallelism
8	4	2	Hybrid parallelism

Run Multi-GPU Inference

torchrun --nproc-per-node=2 examples/wan2.1_fun/predict_t2v.py

5. Additional Resources

Official GitHub: https://github.com/aigc-apps/VideoX-Fun