MOCOV3.md

Experiments on MoCo-v3

Pre-Training

# 1024 batch-sizes on 8 GPUs:
cd projects/mocov3
ssl_train -b 1024 -d 0-7 -e 400 -f ./mocov3_exp.py --amp \
--exp-options exp_name=mocov3/mocov3_tiny_400e

Fine-Tuning

# 1024 batch-sizes on 8 GPUs:
cd projects/eval_tools
ssl_train -b 1024 -d 0-7 -e 300 -f finetuning_exp.py --amp \
[--ckpt <checkpoint-path>] \
--exp-options pretrain_exp_name=mocov3/mocov3_tiny_400e layer_decay=0.75 weights_prefix=base_encoder

Main Results

pre-train code	pre-train epochs	fine-tune recipe	fine-tune epoch	accuracy	ckpt
mocov3	400	-	-	-	link
		impr.	300	76.8	link