Data Preprocessing

October 17, 2025 · View on GitHub

This repository contains Python scripts to process data generated by Unreal for the BEDLAM and BEDLAM2 datasets. The scripts convert the raw data into a format suitable for training models like BEDLAM-CLIFF/BEDLAM-HMR.

BEDLAM v1 Data Processing

This section covers the original BEDLAM dataset.

1. Prepare Data

SMPL-X

  1. Register on the BEDLAM website.
  2. Download the BEDLAM 30fps images (-png) and ground truth camera data (-gt) into a bedlam_data/images directory from the BEDLAM download page.
  3. Register on the SMPL-X website.
  4. Run the following script to download the necessary SMPL-X model files and BEDLAM animation files. 
    bash download_utils.sh

SMPL

  1. Follow steps 1 and 2 from the SMPL-X setup.
  2. Register on the SMPL website.
  3. Run the following script to download the SMPL model and animation files. 
    bash download_utils_smpl.sh

2. Run Processing

Once the data is downloaded, run the appropriate script to generate the output *.npz file for each scene. You can select specific scenes by modifying the bedlam_scene_names.csv file.

  • For SMPL-X data: 
    python df_full_body.py --fps 6
  • For SMPL data: 
    python df_full_body_smpl.py --fps 6

You can generate output at 6fps (every 5th frame) or 30fps (every frame).

Note on Rotated Images

Some scenes, identified by the closeup flag in bedlam_scene_names.csv, contain images rotated by 90 degrees. The processing script handles the ground truth, but you must rotate the image itself for correct projection.

3. Details of Generated .npz Files

The output .npz files contain the following fields:

imgname:      Name of the image
center:       Center of the subject in the image
scale:        Scale of the subject in the image
pose_cam:     SMPL-X/SMPL pose parameters in camera coordinates
pose_world:   SMPL-X/SMPL pose parameters in world coordinates
shape:        10 SMPL-X/SMPL shape components
trans_cam:    Translation of the body in camera coordinates
trans_world:  Translation of the body in world coordinates
gtkps:        2D keypoints
cam_int:      Camera intrinsic matrix
cam_ext:      Camera extrinsic matrix
gender:       Gender of the subject
proj_verts:   Subsampled 2D projected vertices
motion_info:  AMASS motion info and BABEL action label
sub:          Identity of the subject

BEDLAM v1 Data Processing (BEDLAM2 Compatible)

This script processes the original BEDLAM dataset but saves the output in the same format as BEDLAM2. It uses the ground truth motion files with a neutral head model and 16 shape parameters (betas), which can be downloaded from the BEDLAM website (SMPL-X neutral locked head 16 betas ). Download and unzip in bedlam_data/smplx_gt/lockedhead_16betas_motioninfo

1. Run Processing

python df_full_body_bedlam1_compatible.py --fps 6

This will generate .npz files with the same structure as the BEDLAM2 data, including the enhanced annotations and 16 shape parameters.

BEDLAM v2 Data Processing

This section covers the BEDLAM2 dataset, which includes enhanced annotations and uses an updated processing script.

1. Prerequisites

  1. Register on the BEDLAM2 website.
  2. Register on the SMPL-X website.
  3. Register on the SMPL website.
  4. Download and untar the BEDLAM2 30fps images (-png) in bedlam_data/bedlam_v2 and the GT motion file b2_motions_npz_training.tar in bedlam_data/motions_npz_training from the BEDLAM2 download page.
  5. Run the following script to download extra necessary files. 
    bash download_util_bedlam2.sh

2. Run Processing

This script processes the BEDLAM2 dataset, which is defined in bedlam2_scene_names.csv.

python df_full_body_smplx_bedlam2.py --fps 6 --output_folder name_of_output_folder

3. Key Differences and Notes

  • Enhanced Hand Annotations: This script processes and saves bounding box information and detection confidence for both hands.
  • More Shape Parameters: It uses 16 SMPL-X shape parameters (betas) instead of 10.
  • Rotated Images: For BEDLAM2, rotated scenes are identified by the portrait flag in bedlam2_scene_names.csv.
  • Updated SMPL-X Model: It uses a model from smplx_openpose.py to generate joints compatible with the OpenPose format.
  • Removed Head bun (Neutral Head) SMPL-X Model: BEDLAM2 uses neutral head SMPL-X model for rendering instead of v1.1. download_util_bedlam2.sh download the correct model version.

4. Details of Generated .npz Files for BEDLAM2

The .npz files for BEDLAM2 contain additional and modified fields:

lh_c:         Center of the left hand bounding box
rh_c:         Center of the right hand bounding box
lh_s:         Scale of the left hand bounding box
rh_s:         Scale of the right hand bounding box
hand_det_conf: Confidence of left and right hand detections
shape:        16 SMPL-X shape components

Note on gtkps field: The gtkps (2D keypoints) field in BEDLAM2 is a superset containing multiple concatenated joint sets for flexibility. The final array is ordered as follows:

  1. 25 OpenPose-formatted joints: The initial set of joints is mapped to the OpenPose format.
  2. Extra Joints: An additional set of joints generated from a custom regressor.
  3. Original SMPL-X Joints: The full, original joint set from the standard SMPL-X model output.

Note on test subjects: Some of the subjects rendered in training images are made part of test-set and we no longer provide GT labels for them in training. These will be skipped during preprocessing.

Visualization

To help verify the processed data, you can generate visualization images by using a command-line flag when running the processing scripts.

How to Enable Visualization

When running either df_full_body.py or df_full_body_smplx_bedlam2.py, add the --visualize flag followed by your desired visualization type.

For example:

python df_full_body_smplx_bedlam2.py --fps 6 --visualize full

The generated images will be saved in the root directory of the project.

Available Visualization Types

  • --visualize full: Renders the 3D SMPL-X mesh overlayed on the full image.
  • --visualize 2d: Plots the 2D keypoints on the full image.
  • --visualize crop: Renders the 3D mesh on a cropped version of the image based on the subject's bounding box.
  • --visualize none (Default): No visualization images are generated.