CNN-DSO: A combination of Direct Sparse Odometry and CNN Depth Prediction

February 16, 2019 · View on GitHub

This code provides a combination of DSO and Monodepth. For every keyframe, depth values are initialized with the prediction from Monodepth.

Absolute keyframe trajectory RMSE (in meter) on KITTI dataset (DSO and ORB-SLAM numbers are from CNN-SVO paper)

Sequence on KITTI	CNN-DSO	DSO	ORB-SLAM
00	15.13	113.18	77.95
01	5.901	X	X
02	12.53	116.81	41.00
03	1.516	1.3943	1.018
04	0.100	0.422	0.930
05	20.3	47.46	40.35
06	1.547	55.61	52.22
07	8.369	16.71	16.54
08	10.53	111.08	51.62
09	14.00	52.22	58.17
10	4.10	11.09	18.47

Build TensorFlow C++ API (https://github.com/yan99033/monodepth-cpp/tree/master/Tensorflow_build_instructions). This is the hardest part!
Build monodepth-cpp (https://github.com/yan99033/monodepth-cpp).
Prepare Monodepth pre-trained model. You can freaze .ckpt or download the model trained on cityscapes and fine-tuned on kitti here.

Download the repository.

  git clone https://github.com/muskie82/CNN-DSO.git

Modify paths to include directories and libraries of TensorFlow and monodepth-cpp in CMakeLists.txt (4 lines of /abosolute/path/to/XXXXX).

Build

  cd CNN-DSO
  mkdir build
  cd build
  cmake ..
  make -j4

In addition to original DSO command line, you should specify the path to pre-trained model by cnn.

	bin/dso_dataset \
		files=XXXXX/sequence_XX/image_0 \
		calib=XXXXX/sequence_XX/camera.txt \
		cnn=XXXXX/model_city2kitti.pb \
		preset=0 \
		mode=1

Direct Sparse Odometry, Engel, Jakob, Vladlen Koltun, and Daniel Cremers, IEEE transactions on pattern analysis and machine intelligence 40.3 (2018): 611-625. (https://github.com/JakobEngel/dso)
Unsupervised monocular depth estimation with left-right consistency, *Godard, Clément, Oisin Mac Aodha, and Gabriel J. Brostow. *, CVPR. Vol. 2. No. 6. 2017. (https://github.com/mrharicot/monodepth)
CNN-SVO: Improving the Mapping in Semi-Direct Visual Odometry Using Single-Image Depth Prediction., Loo, S. Y., Amiri, A. J., Mashohor, S., Tang, S. H., and Zhang, H, arXiv preprint arXiv:1810.01011 (2018). (https://github.com/yan99033/CNN-SVO)
stereo_dso by HorizonAD: https://github.com/HorizonAD/stereo_dso

GPLv3 license. I don't take any credit from DSO, Monodepth and monodepth-cpp. Please check them.