- All models are trained on keypoint train 2017 images which contains at least one human with keypoint annotations (64115 images).
- The evaluation is done on COCO keypoint val 2017 (5000 images).
- Flip test is used by default.
- One TITAN XP is used for speed test, with
batch_size=64 in each iteration.
- Offline human detection results are used in speed test.
FastPose is our own network design. Paper coming soon!
For example, you can run with:
python scripts/demo_inference.py --cfg configs/halpe_26/resnet/256x192_res50_lr1e-3_1x.yaml --checkpoint pretrained_models/halpe26_fast_res50_256x192.pth --indir examples/demo/ --save_img
- This model is trained based on the first 26 keypoints of Halpe Full-body datatset (without face and hand keypoints).
- The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with
batch_size=64 in each iteration and offline yolov3 human detection results.
For the most accurate wholebody pose estimation, you can run with:
python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-dcn-combined.yaml --checkpoint pretrained_models/multi_domain_fast50_dcn_combined_256x192.pth --indir examples/demo/ --save_img
or, you can run with (this version is a little faster and more accurate on body keypoints, but its performance on hand keypoints is worser):
python scripts/demo_inference.py --cfg configs/halpe_coco_wholebody_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml --checkpoint pretrained_models/multi_domain_fast50_regression_256x192.pth --indir examples/demo/ --save_img
- Above models are trained on multiple datasets, thus they can perform well for in-the-wild images.
- 'Combined (10 hand weight)' means that we use different loss for hand and body keypoints
For example, you can run with:
python scripts/demo_inference.py --cfg configs/halpe_136/resnet/256x192_res50_lr1e-3_2x-regression.yaml --checkpoint pretrained_models/halpe136_fast50_regression_256x192.pth --indir examples/demo/ --save_img
- All of above models are trained only on Halpe Full-body dataset.
- The APs are tested under Halpe's criterion, with flip test on.
- Combined loss means we use heatmap loss (mse loss) on body and foot keypoints and use symmetric integral loss (l1 joint regression loss) on face and hand keypoints.
- There are two FastPose-DCN models with combined loss. The second one uses ten times of weight of hand keypoints, so it is more accurate on hand keypoints but less accurate on the other keypoints.
- The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with
batch_size=64 in each iteration and offline yolov3 human detection results.
- All of above models are trained only on COCO WholeBody dataset.
- The APs are tested under COCO WholeBody's criterion, with flip test on.
- The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with
batch_size=64 in each iteration and offline yolov3 human detection results.
- These models are strongly recommended because they are more accurate and flexible.
- These models are trained with multi-domain knowledge distillation (MDKD, see our paper for more details).
- The APs are tested under Halpe's criterion, with flip test on.
- If you want to use the single hand model, you should give the rough bounding box of a single hand instead of that of a whole person.
- The speed is tested on COCO val2017 on a single NVIDIA GeForce RTX 3090 gpu, with
batch_size=64 in each iteration and offline yolov3 human detection results.
| Model | Backbone | Input Size | PA-MPJPE (3DPW) | PA-MPJPE (Human3.6M) | Download | Config |
|---|
| HybrIK | ResNet34 | 256x256 | 45.3 | 36.3 | model | cfg |