NOTE: the conversion package and this documentation has been adapted from our NSAVP dataset.

We provide scripts for converting rosbag data to H5, H5 to rosbag, and H5 to common formats (CSV and PNG), as well as combining H5 files.

1. Setup

A Docker image for running the code described in this page can be built as follows:

cd /path/to/trnerf/
docker build \
    --build-arg USER_ID=$(id -u) \
    --tag trnerf_conversion \
    --file docker/Dockerfile_conversion \
    .

The result will be a new Docker image named trnerf_conversion with the conversion package built in the /home/user/catkin_ws/ directory within the image.

All commands in subsequent sections are run through an interactive bash session in a Docker container running the trnerf_conversion image. The following commands will run the image and start an interactive bash session in the running container:

xhost +
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v /path/to/trnerf_dataset_folder/:/data trnerf_conversion

The arguments -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix are included to allow display windows to be opened from within the container. The /path/to/trnerf_dataset_folder/ path should be replaced with the folder storing the TRNeRF dataset on your host machine. This folder will appear within the container as the /data folder.

Note that the HDFView GUI is installed in the trnerf_conversion Docker image and can be used to view the contents of H5 files with the following command:

HDFView /path/to/file.h5

2. Converting a ROS1 Rosbag File to H5 Files

The rosbag_to_h5 script can be used to convert a ROS1 rosbag file to multiple H5 files using the strategy described in the General H5 Format section of the data format documentation page.

The script can be run with rosbag_to_h5.launch. The launch file takes the following arguments:

• path_bag_in: A path specifying the ROS1 rosbag file to convert.
• path_h5_folder_out: Path specifying a folder to write the output H5 files to.
• topics: A list of the topics to process in the conversion. If empty, all topics with supported message types will be processed. Defaults to an empty list [].

For example, if there is a rosbag file medium_indoor.bag located in the folder /data/medium_indoor/ within the running Docker container (see Setup) with topics /mono_left/image_raw, /mono_left/image_meta, /mono_right/image_raw, and /mono_right/image_meta, the rosbag can be converted to H5 files with the following command:

roslaunch conversion rosbag_to_h5.launch path_bag_in:=/data/medium_indoor/medium_indoor.bag path_h5_folder_out:=/data/medium_indoor/

The result would be two new H5 files, mono_left.h5 and mono_right.h5, located in the folder /data/medium_indoor/, each containing /image_raw and /image_meta groups.

3. Combining H5 Files

The h5_combine script can be used to symbollically and directly combine H5 files. In the symbolic case, a new H5 file is created with external links to the original H5 files without copying any data. In the direct case, a new H5 file is created and data from the original H5 files is copied into it. The resulting files are functionally equivalent, as long as the external link filepaths remain valid in the symbolic case. In both cases, the root groups in the original H5 files appear as top level groups in the new H5 file with names derived from the filenames of the original H5 files.

The script can be run with h5_combine.launch. The launch file takes the following arguments:

• path_base_folder: A path specifying a base folder that all other paths are appended to. If this path is given and the filepaths_h5_in list is empty, the script will combine all .h5 files in the top level of the path_base_folder folder. Defaults to an empty string "".
• filepath_h5_out: A filepath for the new H5 file to be created. This path is appended to path_base_folder. Defaults to an empty string "".
• filepaths_h5_in: A list of filepaths specifying the original H5 files to be combined. These paths are appended to path_base_folder. Defaults to an empty list [].
• symbolic: A boolean specifying whether (true) to symbolically combine the original H5 files using external links or (false) copy the data from the original H5 files into the new file. Defaults to true.
• relative: A boolean specifying whether (true) to create external links with relative filepaths (i.e. relative to the folder containing filepath_h5_out) or (false) absolute filepaths. Ignored if symbolic is false. Defaults to true.

For example, if the files adk_right.h5 and imu.h5 are located in the folder /data/medium_indoor/ within the running Docker container (see Setup) they can be combined with the following command:

roslaunch conversion h5_combine.launch path_base_folder:=/data/medium_indoor/ filepath_h5_out:=medium_indoor.h5

The result would be a new H5 file, /data/medium_indoor/medium_indoor.h5, containing groups /adk_right and /imu.

In this example, the groups are external links with relative paths. If the combined H5 file is moved, the links will no longer point to the files. Absolute paths can instead be used with the argument relative:=false, but note that the absolute paths would be within the Docker container's filesystem in this example. Another option is to use the argument symbolic:=false, in which case the original data will be copied and the new H5 file will contain no external links.

4. Converting H5 Files to a ROS1 Rosbag File

The h5_to_rosbag script can be used to convert a H5 file to a ROS1 rosbag file. To use the script, first combine the H5 files to be converted using h5_combine, as described above in Combining H5 Files, and then run this script on the resulting H5 file. Note that h5_combine must be used even if converting a single H5 file (e.g. adk_right.h5) because the h5_to_rosbag script assumes the top level groups in the H5 file correspond to ROS namespaces (e.g. /adk_right) and that subgroups correspond to topics (e.g. /adk_right/image_raw). The script additionally assumes the subgroups are structured as described in the General H5 Format section of the data format documentation page.

The script can be run with h5_to_rosbag.launch. The launch file takes the following arguments:

• path_h5_in: A path specifying the H5 file to convert.
• path_bag_out: A path specifying the rosbag to write.
• topics: A list of topics to process in the conversion. The ROS topic names match the subgroup paths in the input H5 file (e.g. /adk_right/image_raw). If empty, all topics with supported message types will be processed. Defaults to an empty list [].

Continuing the example given above in Combining H5 Files, the medium_indoor.h5 file located in the folder /data/medium_indoor/ within the running Docker container (see Setup) can be converted to a rosbag with the following command:

roslaunch conversion h5_to_rosbag.launch path_h5_in:=/data/medium_indoor/medium_indoor.h5 path_bag_out:=/data/medium_indoor/medium_indoor.bag

The result would be a new rosbag file, /data/medium_indoor/medium_indoor.bag, containing a topic for each subgroup in medium_indoor.h5 (/adk_right/image_raw, /adk_right/image_meta, etc.).

5. Converting H5 Files to Common Formats (CSV and PNG)

The h5_to_common script can be used to convert a H5 file to common formats (CSV and PNG files). To use the script, first combine the H5 files to be converted using h5_combine, as described above in Combining H5 Files, and then run this script on the resulting H5 file. Note that h5_combine must be used even if converting a single H5 file (e.g. adk_right.h5) because the h5_to_common script assumes the top level groups in the H5 file correspond to ROS namespaces (e.g. /adk_right) and that subgroups correspond to topics (e.g. /adk_right/image_raw). The script additionally assumes the subgroups are structured as described in the General H5 Format section of the data format documentation page.

The result of the h5_to_common script is a folder for each namespace (e.g. adk_right/) that contains a file or subfolder for each processed topic (e.g. the adk_right/ folder would contain the folder image_raw/ and the file image_meta.csv). Image topics are output as a folder of PNG files named as <timestamp>.png where <timestamp> is the integer timestamp of the image in nanoseconds. All other topics are output as CSV files with a column (or multiple columns) for each H5 dataset (e.g. the imu.csv file contains the columns timestamp, x_angular_velocity, y_angular_velocity, etc.). The first row of these CSV files is a header listing the dataset (or dataset component) name corresponding to each column. Details such as units are as described in the Data Format documentation page (e.g. all timestamps are in nanoseconds, etc.).

The script can be run with h5_to_common.launch. The launch file takes the following arguments:

• path_h5_in: A path specifying the H5 file to convert.
• path_folder_out: A path specifying the ouput folder which will contain the CSV files and/or subfolders of PNG images.
• topics: A list of topics to process in the conversion. The ROS topic names match the subgroup paths in the input H5 file (e.g. /adk_right/image_raw). If empty, all topics with supported message types will be processed. Defaults to an empty list [].

Continuing the first example given above in Combining H5 Files, the medium_indoor.h5 file located in the folder /data/medium_indoor/ within the running Docker container (see Setup) can be converted to common formats with the following command:

roslaunch conversion h5_to_common.launch path_h5_in:=/data/medium_indoor/medium_indoor.h5 path_folder_out:=/data/medium_indoor/medium_indoor_common/

The result would be a new folder, /data/medium_indoor/medium_indoor_common/, containing subfolders for each group in medium_indoor.h5, adk_right/ and imu/, with adk_right/ containing the subfolder image_raw/ and file image_meta.csv and imu/ containing the files imu.csv, magnetic_field.csv, fluid_pressure.csv, and temperature.csv.