Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework

This repo covers an reference implementation for the paper Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework in PyTorch, using Deep Fashion In-Store as an illustrative example: Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework.

Requirements

• Pytorch 1.7.0
• tensorboard_logger 0.1.0

Typo

• We want to correct a typo in the paper. In Eq. 6 and 7, should be .

Running

• This code is built upon two codebases: Supervised Contrastive Learning and MoCo.

• Train pre-trained model on Deep Fashion In-store dataset

  • Perpare train-listfile, val-listfile. The format is as follows:

  {
    "images": [
      "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_1_front.jpg",
      "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_2_side.jpg",
      "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_4_full.jpg",
      "/deep_fashion_in_store/img/WOMEN/Dresses/id_00000002/02_7_additional.jpg",
      "/deep_fashion_in_store/img/WOMEN/Blouses_Shirts/id_00000004/03_1_front.jpg"
    ],
    "categories": [
      "Dresses",
      "Dresses",
      "Dresses",
      "Dresses",
      "Blouses_Shirts"
    ]
  }
  
  

  • Class map can be downloaded from class_map.json

  • Repeating product ids can be downloaded from repeating_product_ids.csv

  • If experiment on the model transfer ability from seen classes to unseen classes, the two classes maps can be downloaded from class_map_seen.json and class_map_unseen.json.

  • To train the model on Deep Fashion In-store dataset, run

  python train_deepfashion.py --data ./deepfashion/ 
  --train-listfile ./train_listfile.json 
  --val-listfile ./val_listfile.json 
  --test-listfile ./test_listfile.json 
  --class-map-file ./classmap.json 
  --num-classes 17 
  --learning_rate 0.5 --temp 0.1
  --ckpt /pretrained_model/
  --dist-url 'tcp://localhost:10001' 
  --multiprocessing-distributed 
  --world-size 1 --rank 0 --cosine
  
  

  • To evaluate the model, run

  python eval_deepfashion.py --data ./deepfashion/ 
  --train-listfile ./train_listfile.json --val-listfile ./val_listfile.json 
  --class-map-file ./classmap.json 
  --num-classes 17 
  --learning_rate 0.5 --temp 0.1
  --ckpt /trained_model/
  
  

Reference

@inproceedings{hierarchicalContrastiveLearning,
      title={Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework}, 
      author={Shu Zhang and Ran Xu and Caiming Xiong and Chetan Ramaiah},
      year={2022},
      booktitle={CVPR},
}