RGB2Dcaffecpp

June 13, 2018 ยท View on GitHub

Get depth map from one web camera using caffe framework with cpp and using it to create 3D and SLAM.

Compile and run:

  1. Create a folder called cpp_depth/ in ./caffe/examples/, copy all the files to ./caffe/examples/cpp_depth/
  2. Create a folder called depth_model/ in ./caffe/models/ and copy .caffemodel and .protxt file into it(.caffemodel file is too big to upload to github, please download it here: Link).
  3. Build the caffe again:
$ cd build/
$ cmake ..
$ make
  1. If you have a camera and need to run in real time, connect your webcamera to the PC and in caffe root directory run:
cd /media/elab/sdd/mycodes/caffe

./build/examples/cpp_depth/depth_camera.bin   \
./models/depth_model/model_norm_abs_100k.prototxt  \
 ./models/depth_model/model_norm_abs_100k.caffemodel

or you can also using directly the python script(now in caffe/):

python ./examples/cpp_depth/depth_camera.py
  1. If you have a folder of RGB images and need to create depth images from that(change the input and output folder path in the script):
cd ..
python ./examples/cpp_depth/depth_images.py

Run on TX1/2

If you want it to work on TX1/2, you have to install caffe for TX1: https://github.com/jetsonhacks/installCaffeJTX1, and then do the same thing above.

Create 3D and SLAM from RGB and Depth images

  1. Make and install [RTABmap].
  2. Put the rgb images in /rgb_sync, the depth images in /depth_sync, and run to create rtabmap.db and rtbmap_poses.txt:
cd /media/enroutelab/sdd/mycodes/rtabmap/bin
./rtabmap-rgbd_dataset        --Vis/EstimationType 1       --Vis/BundleAdjustment 1       --Vis/PnPReprojError 1.5       --Odom/GuessMotion true       --OdomF2M/BundleAdjustment 1       --Rtabmap/CreateIntermediateNodes true       --Rtabmap/DetectionRate     <IMAGEFOLDER>

Example for the TUM data:

 ./rtabmap-rgbd_dataset        --Vis/EstimationType 1       --Vis/BundleAdjustment 1       --Vis/PnPReprojError 1.5       --Odom/GuessMotion true       --OdomF2M/BundleAdjustment 1       --Rtabmap/CreateIntermediateNodes true       --Rtabmap/DetectionRate     /media/enroutelab/sdd/data/tum_depth/rgbd_dataset_freiburg3_long_office_household
  1. Run ./rtabmap -> Tools -> Edit database -> Load rtabmap.db created.
  2. Edit -> Regenerate local grid maps
  3. View -> 3D View