LiDARInertialVisual_Handhold

May 19, 2025 · View on GitHub

News

  • Apr. 04, 2025 - Add support for multiple cameras.
  • Mar. 24, 2025 - Open-sourced the UAV platform with hard-sync LIV sensors used in FAST-LIVO2. 🎉
  • Oct. 27, 2024 - Add modified livox_ros_driver2 with time sync support for Mid360.
  • Oct. 11, 2024 - Add support for other pixel formats besides RGB8 (e.g., BayerRG8, BayerRG12Packed).
  • Oct. 03, 2024 - Add ARM support, fix mvs_ros_driver startup and shutdown bug, remove the SerialNumber dependency, resolve the auto-exposure mode switching issue, and add a TriggerEnable interface to allow testing the camera without the PPS signal.
  • Jul. 15, 2024 - Fix the bug related to LiDAR (IMU) timestamp compensation in the livox_ros_driver, remove the dynamic library dependency from mvs_ros_driver, and enable this synchronization scheme to use the official livox_sdk.
  • Jun. 28, 2024 - Update reproduction videos and some important notes.
  • May. 27, 2024 - Fix a bug in the calculation of the synthetic GPRMC timestamp, which could cause timestamp rollback. Additionally, we add checksum verification for GPRMC.
  • May. 20, 2024 - According to Livox Avia's pin requirements, we convert PPS TTL level to RS485 level and supplement it with more detailed electronic connection and material list.

📬 For further assistance or inquiries, please feel free to contact Chunran Zheng at zhengcr@connect.hku.hk.

1. Introduction

This repository provides the CAD files (with suffix “*.SLDPRT and *.SLDASM”) for our handheld and uav-mounted devices, which can be opened and edited with Solidworks. All of the modules are suitable for printing with FDM (Fused Deposition Modeling). In addition, we have also open-sourced our hardware synchronization scheme, as well as the STM32 source code and hardware wiring configuration instructions.

2. Relevant guidance video

Thanks to the Bilibili uploader GundaSmart for making a detailed reproduction video. Everyone can follow the video to reproduce it. His sensor configuration includes a MID360 and a Hikvision pinhole camera.

FAST-LIVO Reproduction (Part 1)

FAST-LIVO Reproduction (Part 2)

FAST-LIVO Reproduction (Part 3)

FAST-LIVO Reproduction (Part 4)

FAST-LIVO Reproduction (Part 5)

3. Guide to installation

3.1 Root directory

├── handhold_cad/ - CAD source files for Handheld device
│   ├── ...
├── drone_cad/ - CAD source files for UAV device
│   ├── ...
├── livox_ros_driver/ - Livox LiDAR ROS driver
│   ├── ...
├── mvs_ros_driver/ - Camera driver
│   ├── ...
└── stm32_timersync-open/ - Embedded engineering folder
│   ├── USER/ - Main functionality folder
│   ├── ...
└── README.md - Project homepage document
└── ...

3.2 Assembly instruction

The assembly instructions are demonstrated in the GIFs below. Each module is clearly labeled corresponding to the name of its STL file.

1 2

3.3 Electronic connection

The guide for the electronic connections is presented as follows:

Livox AVIA LiDAR M12 Peripheral Function Diagram
PIN 1 Power: + Diagram
AVIA 12-Pin Interface, top: male bottom: female
PIN 2 Ground
PIN 7 Ethernet: RX-
PIN 6 Ethernet: RX+
PIN 5 Ethernet: TX-
PIN 4 Ethernet: TX+
PIN 11 (Sync-) RS485_Output B-
PIN 12 (Sync+) RS485_Output A+
MVS Camera 6PIN Name I/O Type Description Peripheral Function Diagram
PIN 1 DC_PWR -- Power Supply Diagram
MVS Camera 6-Pin Interface
PIN 2 OPTO_IN Line 0+ Optical Isolation Input STM32 PA1
PIN 3 GPIO Line 2+ General Purpose Input/Output
PIN 4 OPTO_OUT Line 1+ Optical Isolation Output
PIN 5 OPTO_GND Line 0- / 1- Optical Isolation Ground STM32 GND
PIN 6 GND Line 2- Ground
STM32 Peripheral Function
PA1 MVS camera PIN2 (OPTO_IN)
PB5 RS485_Input (TXD)
PA9 TTL2USB (RXD)
VCC RS485_Input (VCC) | TTL2USB (VCC)
GND RS485_Input (GND) | TTL2USB (GND) | MVS camera PIN5 (GND)
RS485 Peripheral Function
RS485_Output A+ LiDAR M12 PIN12 (Sync+)
RS485_Output B- LiDAR M12 PIN11 (Sync-)
RS485_Output (GND) LiDAR M12 PIN2 (GND)

🔥 IMPORTANT:

  • STM32 PB5 (PPS signal) is converted from TTL to RS-485, resulting in RS485_Output A+ and RS485_Output B-;
  • STM32 TXD (GPRMC) is converted from TTL to USB and sent to the PC;
  • If you are using the Mid360, you can directly connect STM32 PB5 to LiDAR M12 pps interface;
  • Before you roslaunch livox_ros_driver livox_lidar_msg.launch, you can use sudo chmod a+rw /dev/ttyUSB0 grant permissions to the USB serial port;
  • The default connection method in this repository directly connects to M12 without using the Livox converter. If you use the converter, you can connect STM32 PB5 (PPS signal) to the Livox converter Sync Port without converting TTL to 485 level. Refer to Issue 19 for details.

4. How to run the driver

4.1 Prerequisites

Download and install MVS-2.1.2 and Livox-SDK.

4.2 Execution Steps

  1. Launch the MVS camera driver with trigger mode:
roslaunch mvs_ros_driver mvs_camera_trigger.launch
  1. Start the Livox LiDAR driver:
roslaunch livox_ros_driver livox_lidar_msg.launch
  1. Record sensor data:
rosbag record /livox/lidar /livox/imu left_camera/image

5. Bill of materials (only for reference)

ItemPicsPurchasing list
Livox Avia LiDARLivox Avia
CMOSMV-CA013-21UC
Camera Len MVL-HF0628M-6MPE
STM32STM32F103C8T6
ScreenIPS Screen 10''
Battery4800mah
TTL to USBTTL to USB
TTL to 485TTL to 485

6. Alternative UAV Platform

For detailed equipment drawings, refer to the drone_cad folder.

7. License

The source code is released under GPLv3 license.

If you use any code of this repo in your academic research, it will be very appreciated if you can cite any of our following papers:

[1] Zheng, Chunran, et al. "FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry." 
[2] Zheng, Chunran, et al. "FAST-LIVO: Fast and Tightly-coupled Sparse-Direct LiDAR-Inertial-Visual Odometry." 
[3] Hong, Sheng, et al. "Rollvox: real-time and high-quality LiDAR colorization with rolling shutter camera."