mincd-pnp-demo

a demo implementation of mincd-pnp

we upload the code in the file of new_files_add. Our file is based on MATR (ICCV'23). This source code can be searched in Github. The core implementation of MinCD is in lines 154-301 of model.py. It has two stages:

(1) stage one lines 162-188, detects 2d and 3d keypoints, and align them via L2 feature distances (2) stage two lines 193-296, regress the camera pose from the 2d/3d keypoints, and reproject 3d keypoints into 2d image plane with the predicted camera pose. loss is computed to minimize the distance between 2d keypoints and 3d

Q1: how to generate 2d keypoints A1: please see the anyscene_dataset_kpts in the github/mincd-pnp-demo, 2d keypoints are generated by opencv gftt api. It is loaded in model.py at line 166 as "image_mask"

Q2: where is the loss function? A2: please see the loss file in the github/mincd-pnp-demo. the loss is the reprojection loss.

Q3: how to do two-stage training? A3: it is automatically implemented. In line 193 of model.py, you can see that the preparation of chamfer loss will be computed after the epoch of 20.

We strongly recommand you to setup and run MATR at first. After understanding the code structure of MATR, it is easy to implement MinCD-PnP. If there is any problem, feel free to contact me or send message in Github.