Halpe Full-Body Human Keypoints and HOI-Det dataset

What is Halpe?

Halpe is a dataset introduced in AlphaPose paper. It aims at pushing Human Understanding to the extreme. We provide detailed annotation of human keypoints, together with the human-object interaction trplets from HICO-DET. For each person, we annotate 136 keypoints in total, including head,face,body,hand and foot. Below we provide some samples of Halpe dataset.

Download

Train annotations [Baidu | Google ]

Val annotations [Baidu | Google ]

Train images from HICO-DET

Val images from COCO

Realtime Demo with tracking

Trained model is available in AlphaPose! Check out its MODEL_ZOO

Keypoints format

We annotate 136 keypoints in total:

    //26 body keypoints
    {0,  "Nose"},
    {1,  "LEye"},
    {2,  "REye"},
    {3,  "LEar"},
    {4,  "REar"},
    {5,  "LShoulder"},
    {6,  "RShoulder"},
    {7,  "LElbow"},
    {8,  "RElbow"},
    {9,  "LWrist"},
    {10, "RWrist"},
    {11, "LHip"},
    {12, "RHip"},
    {13, "LKnee"},
    {14, "Rknee"},
    {15, "LAnkle"},
    {16, "RAnkle"},
    {17,  "Head"},
    {18,  "Neck"},
    {19,  "Hip"},
    {20, "LBigToe"},
    {21, "RBigToe"},
    {22, "LSmallToe"},
    {23, "RSmallToe"},
    {24, "LHeel"},
    {25, "RHeel"},
    //face
    {26-93, 68 Face Keypoints}
    //left hand
    {94-114, 21 Left Hand Keypoints}
    //right hand
    {115-135, 21 Right Hand Keypoints}

Illustration:

26 body keypoints

68 face keypoints

21 hand keypoints

Usage

The annotation is in the same format as COCO dataset. For usage, a good start is to check out the vis.py. We also provide related APIs. See halpecocotools, which can be installed by pip install halpecocotools.

Evaluation

We adopt the same evaluation metrics as COCO dataset.

Related resources

A concurrent work COCO-WholeBody also annotate the full body keypoints. And HOI-DET for COCO dataset is also available at V-COCO

Citation

If the data helps your research, please cite the following paper:

@article{alphapose,
  author = {Fang, Hao-Shu and Li, Jiefeng and Tang, Hongyang and Xu, Chao and Zhu, Haoyi and Xiu, Yuliang and Li, Yong-Lu and Lu, Cewu},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  title = {AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time},
  year = {2022}
}