VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data

We present VoxelKP, a novel fully sparse network architecture tailored for human keypoint estimation in LiDAR data. We achieved the state-of-the-art performances without additional data.

Performance Demo

Installation

Please refer to the installation of OpenPCDet.

Getting Started

Please refer to GETTING_STARTED.md to learn more usage about this project.

This is the inference only code.

Checkpoints can be downloaded from here.

$ cd tools
$ python waymo_visualizer.py --ckpt CHECKPOINT

Train

1. Prepare dataset, referring to OpenPCDet.

2. Install the module for compling.

$ pip install -e .

3. The training script.

$ cd tools
$ python train.py  --cfg_file ./cfgs/waymo_models/kp_effv2next4_voxelnext_iou_aug_bev_channel.yaml --epochs 20

if using multiple GPUs

$ bash scripts/dist_train.sh 8 --cfg_file ./cfgs/waymo_models/kp_effv2next4_voxelnext_iou_aug_bev_channel.yaml --epochs 36 --workers 0

Benchmarks

There is a limited number of relevant research for this task. Most of the prior works utilize additional training data beyond the 3D keypoint data within the Waymo dataset. To provide a fair comparison, we need to consider approaches that use extra data and those that rely solely on Waymo ground truth separately.

Experiment Results

We report the full spectrum of the evaluation, including MPJPE, OKS@AP, and PEM.

Part	MPJPE	OKS@KP	PEM
Head	0.0570	0.6393	0.1569
Shoulders	0.0669	0.8917	0.1563
Elbows	0.0948	0.7197	0.1746
Wrists	0.1467	0.3791	0.1987
Hips	0.0670	0.9533	0.1576
Knees	0.0820	0.8586	0.1660
Ankles	0.1084	0.7581	0.1765
All	0.0887	0.7300	0.1695

Our visual results show that our VoxelKP offers improved keypoint estimation with precise locations and fewer false positives.

A visual demonstration of our baseline model (top) and the proposed VoxelKP (bottom). The insets are color-coded according to the legend in the figure. In the green-colored insets, a comparison with the ground truth is shown, with ground truth in red and predictions in blue.

Architecture

Cite our work

@inproceedings{shi2025voxelkp,
  title={VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data},
  author={Shi, Jian and Wonka, Peter},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={28282--28291},
  year={2025}
}

Acknowledgement

This repository is built on top of OpenPCDet and VoxelNeXt.

We use sptr implementation from here.