INSTALL.md

Installation

This repo was tested with Python 3.6, PyTorch 1.1.0, and CUDA 9.0. But it should be runnable with recent PyTorch versions >=1.0.0. (0.4.x may be also ok)

python setup.py develop # OR python setup.py install

Preparation

Datasets

Pittsburgh, Tokyo 24/7 and Tokyo Time Machine datasets can be found HERE. The associated .mat files are available HERE.

mkdir data

Download the raw datasets and then unzip them under the directory like

data
├── pitts
│   ├── raw
│   │   ├── pitts250k_test.mat
│   │   ├── pitts250k_train.mat
│   │   ├── pitts250k_val.mat
│   │   ├── pitts30k_test.mat
│   │   ├── pitts30k_train.mat
│   │   ├── pitts30k_val.mat
│   │   ├── Pittsburgh/
│   │   │   ├── images
│   └── └── └── queries
│
└── tokyo
    ├── raw
    │   ├── tokyo247/
    │   ├── tokyo247.mat
    │   ├── tokyoTM/
    │   ├── tokyoTM_train.mat
    └── └── tokyoTM_val.mat

Use Custom Dataset (Optional)

1. Download your own dataset and save it under

data/my_dataset
  └── raw/ # save the images here

2. Define your own dataset following the template, and save it under haf/datasets/, e.g. haf/datasets/my_dataset.py.

3. Register it in haf/datasets/__init__.py, e.g.

from .my_dataset import MyDataset # MyDataset is the class name
__factory = {
    'my_dataset': MyDataset,
}

4. (Optional) Read it by

from haf.datasets import create
dataset = create('my_dataset', 'data/my_dataset') # you can use this command for debugging

5. Use it for training/testing by adding args of -d my_dataset in the scripts.

Pre-trained Weights

mkdir logs && cd logs

After preparing the pre-trained weights, the file tree should be

logs
├── haf_vgg16_conv_1_dim-[64/128/384].pth # refer to (1)
├── haf_vgg16_conv_m_dim-[64/128/256].pth # refer to (1)
├── haf_vgg16_conv_h_dim-[64/128/512].pth # refer to (1)
└── vgg16_pitts_64_desc_cen_[64/128/384]_[64/128/256]_[64/128/512].hdf5 # refer to (2)

(1) Architecture-pretrained weights for VGG16 backbone

Our VGG16-based feature extraction layers is pretrained on Architecture dataset. Directly download from Google Drive, and save them under the path of logs/.

Or you can train the feature branch by running the Feature_branch.py (You should download the Architecture dataset and put them under the path of Arch_test and Arch_train):

# for small model (for one V100 GPU)
bash run_Feature_branch.sh

Note: the models and results here are trained by this repo. In order to accomodate computation efficiency, we use small channel dimensions (64*3) and feature sizes of 128/2 X 128/2. We encourage you to change the channel dimensions (L_DIM, M_DIM, H_DIM) in run_Feature_branch.sh to increase the performance.

(2) initial cluster centers for VLAD layer

Note: it is important as the VLAD layer cannot work with random initialization.

The original cluster centers provided by NetVLAD are highly recommended. You could directly download from Google Drive and save it under the path of logs/.

Or you could compute the centers by running the script (if you change the channel dimensions, you must recompute the centers):

bash cluster.sh