High Speed and High Dynamic Range Video with an Event Camera

August 24, 2026 · View on GitHub

Note: Modernized in 2026 for Python/PyTorch setup with uv and using jAERas the simplest way to evaluate E2VID or FireNet on recorded files or a live event camera.

High Speed and High Dynamic Range Video with an Event Camera

This is the code for the paper High Speed and High Dynamic Range Video with an Event Camera by Henri Rebecq, Rene Ranftl, Vladlen Koltun and Davide Scaramuzza:

You can find a pdf of the paper here. If you use any of this code, please cite the following publications:

@Article{Rebecq19pami,
  author        = {Henri Rebecq and Ren{\'{e}} Ranftl and Vladlen Koltun and Davide Scaramuzza},
  title         = {High Speed and High Dynamic Range Video with an Event Camera},
  journal       = {{IEEE} Trans. Pattern Anal. Mach. Intell. (T-PAMI)},
  url           = {https://rpg.ifi.uzh.ch/docs/TPAMI19_Rebecq.pdf},
  year          = 2019
}

@Article{Rebecq19cvpr,
  author        = {Henri Rebecq and Ren{\'{e}} Ranftl and Vladlen Koltun and Davide Scaramuzza},
  title         = {Events-to-Video: Bringing Modern Computer Vision to Event Cameras},
  journal       = {{IEEE} Conf. Comput. Vis. Pattern Recog. (CVPR)},
  year          = 2019
}

Install

Requires uv and Python 3.13. From this repo, pick one extra:

uv sync --extra cpu      # CPU PyTorch
uv sync --extra cu128    # NVIDIA CUDA PyTorch
uv sync --extra xpu      # Intel Arc PyTorch (same torch API, xpu device)

Pick one extra. The package is still torch; only the wheel index changes. Scripts already default --use_gpu. Device order is CUDA, then Intel XPU, then CPU. You should see Device: cuda:0 or Device: xpu.

--extra cu128 does not install the NVIDIA CUDA Toolkit and does not change the display driver. It only downloads a PyTorch wheel that already contains the CUDA runtime.

--extra xpu is the Intel Arc path (PyTorch XPU). Use a current Arc driver (this laptop already has one). The XPU runtime comes with the wheel; you do not need a separate Intel Deep Learning Essentials install for the binary wheel.

Linux

curl -LsSf https://astral.sh/uv/install.sh | sh
source "$HOME/.local/bin/env"
uv python install 3.13
cd /path/to/rpg_e2vid
uv sync --extra cpu      # or: uv sync --extra cu128  or: uv sync --extra xpu

Windows

If uv or Python 3.13 are not installed yet:

winget install --id Python.Python.3.13 -e --accept-package-agreements --accept-source-agreements
winget install --id astral-sh.uv -e --accept-package-agreements --accept-source-agreements

Open a new PowerShell, then:

cd \path\to\rpg_e2vid
uv sync --extra cpu      # or: uv sync --extra cu128  or: uv sync --extra xpu

If the repo is under Dropbox, put the venv outside it (avoids file locks and a multi-GB wheel being synced):

$env:UV_PROJECT_ENVIRONMENT = "$env:USERPROFILE\.venvs\rpg-e2vid-xpu"
uv sync --extra xpu      # this Intel Arc laptop; use cu128 on NVIDIA
# later runs must set the same variable, or uv will use the in-repo .venv (CPU / Dropbox locks)

Run

Download either or both pretrained networks into pretrained/. jAER is the simplest way to evaluate them on recorded files or a live camera (see Live from jAER).

On an NVIDIA GeForce RTX 2080 SUPER, total per-frame time is about 6 ms for FireNet and 31 ms for E2VID_lightweight. Power consumption (nvidia-smi) is static ~56W, active ~74W.

Linux / macOS (E2VID plus an optional example event zip):

mkdir -p pretrained data
curl -L -o pretrained/E2VID_lightweight.pth.tar "https://rpg.ifi.uzh.ch/data/E2VID/models/E2VID_lightweight.pth.tar"
curl -L -o data/dynamic_6dof.zip "https://rpg.ifi.uzh.ch/data/E2VID/datasets/ECD_IJRR17/dynamic_6dof.zip"

Windows (PowerShell; curl.exe is built in):

New-Item -ItemType Directory -Force -Path pretrained, data | Out-Null
curl.exe -L -o pretrained/E2VID_lightweight.pth.tar "https://rpg.ifi.uzh.ch/data/E2VID/models/E2VID_lightweight.pth.tar"
curl.exe -L -o data/dynamic_6dof.zip "https://rpg.ifi.uzh.ch/data/E2VID/datasets/ECD_IJRR17/dynamic_6dof.zip"

FireNet (optional; same runner): download the FireNet weights and save them as pretrained/firenet_1000.pth.tar.

Choose the network with --network e2vid or --network firenet (sets the default -c path); -c still overrides. The checkpoint arch is printed at startup and in the timing summary.

Linux / macOS reconstruction:

uv run python run_reconstruction.py --network e2vid -i data/dynamic_6dof.zip --auto_hdr --display --show_events
uv run python run_reconstruction.py --network firenet -i data/dynamic_6dof.zip --auto_hdr --display --show_events

Windows (PowerShell):

uv run python run_reconstruction.py --network e2vid -i data/dynamic_6dof.zip --auto_hdr --display --show_events
uv run python run_reconstruction.py --network firenet -i data/dynamic_6dof.zip --auto_hdr --display --show_events

Headless (write frames, no OpenCV window) — same on Linux and Windows:

uv run python run_reconstruction.py --network e2vid -i data/dynamic_6dof.zip --auto_hdr -o output

Live from jAER

jAER is the simplest way to evaluate E2VID or FireNet on recorded files or a live camera. DNNOutputViaSharedMemory (outputMode=EventWindows) publishes packed (t, x, y, p) windows over a mmap file plus localhost TCP (127.0.0.1:14101). Do not use EventCountFrames$ (64 \times 64 \text{uint8} \text{histograms}) \text{or} $ROSOutput for this path. Open it from File → Remote → DNN shared memory output….

  1. In jAER, enable DNNOutputViaSharedMemory and set outputMode to EventWindows. Leave eventsPerWindow=0 to match E2VID (N = width × height × 0.35). Leave flipY checked so exported y=0 is the top of the image (Python / OpenCV). Uncheck it only if you want jAER lower-left coordinates.
  2. Play a live camera or an AEDAT / DSEC file.
  3. Run:
uv run python live_reconstruction.py --network e2vid --auto_hdr --display --show_events
uv run python live_reconstruction.py --network firenet --auto_hdr --display --show_events

--jaer-tcp 127.0.0.1:14101 is the default. --jaer-mmap overrides the path from the HELLO line. Voxel grids are built in Python with the same bilinear binning as run_reconstruction.py. --no-voxel-grid-voting hard-bins polarity counts instead (no split across time slices); toggle with v in the OpenCV window.

Click the OpenCV window so keys reach it:

  • ESC / x quit, r reset recurrent state, ? help, p print settings
  • n recurrent on/off, h auto-HDR, [ ] Imin, ; ' Imax, u i unsharp
  • v voxel-grid voting on/off, e event preview

If Python is slower than jAER, the 2-slot mmap is reused and those windows are overwritten (lost). The live reader skips queued notifies and reconstructs only the latest fresh slot (~15 Hz max on this 640×480 FireNet path). Prints at most every 2 s; summary on exit. To reconstruct every window, close slices slower than that: jAER eventWindowTimeSliceMethod=TimeIntervalUs at ≥80 ms, or a larger eventsPerWindow, and play files in realtime.

Parameters

Below is a description of the most important parameters:

Main parameters

  • --window_size / -N (default: None) Number of events per window. This is the parameter that has the most influence of the image reconstruction quality. If set to None, this number will be automatically computed based on the sensor size, as N = width * height * num_events_per_pixel (see description of that parameter below). Ignored if --fixed_duration is set.
  • --fixed_duration (default: False) If True, will use windows of events with a fixed duration (i.e. a fixed output frame rate).
  • --window_duration / -T (default: 33 ms) Duration of each event window, in milliseconds. The value of this parameter has strong influence on the image reconstruction quality. Its value may need to be adapted to the dynamics of the scene. Ignored if --fixed_duration is not set.
  • --Imin (default: 0.0), --Imax (default: 1.0): linear tone mapping is performed by normalizing the output image as follows: I = (I - Imin) / (Imax - Imin). If --auto_hdr is set to True, --Imin and --Imax will be automatically computed as the min (resp. max) intensity values.
  • --auto_hdr (default: False) Automatically compute --Imin and --Imax. Disabled when --color is set.
  • --color (default: False): if True, will perform color reconstruction as described in the paper. Only use this with a color event camera such as the Color DAVIS346.

Output parameters

  • --output_folder: path of the output folder. If not set, the image reconstructions will not be saved to disk.
  • --dataset_name: name of the output folder directory (default: 'reconstruction').

Display parameters

  • --display (default: False): display the video reconstruction in real-time in an OpenCV window.
  • --show_events (default: False): show the input events side-by-side with the reconstruction. If --output_folder is set, the previews will also be saved to disk in /path/to/output/folder/events.

Additional parameters

  • --num_events_per_pixel (default: 0.35): Parameter used to automatically estimate the window size based on the sensor size. The value of 0.35 was chosen to correspond to ~ 15,000 events on a 240x180 sensor such as the DAVIS240C.
  • --no-normalize (default: False): Disable event tensor normalization: this will improve speed a bit, but might degrade the image quality a bit.
  • --no-recurrent (default: False): Disable the recurrent connection (i.e. do not maintain a state). For experimenting only, the results will be flickering a lot.
  • --voxel-grid-voting (default: True): Bilinear temporal voting between voxel slices. --no-voxel-grid-voting hard-bins polarity counts into slices (event-count histograms). Toggle live with v.
  • --hot_pixels_file (default: None): Path to a file specifying the locations of hot pixels (such a file can be obtained with this tool for example). These pixels will be ignored (i.e. zeroed out in the event tensors).

Example datasets

We provide a list of example (publicly available) event datasets to get started with E2VID.

Working with ROS

Because PyTorch recommends Python 3 and ROS is only compatible with Python2, it is not straightforward to have the PyTorch reconstruction code and ROS code running in the same environment. To make things easy, the reconstruction code we provide has no dependency on ROS, and simply read events from a text file or ZIP file. We provide convenience functions to convert ROS bags (a popular format for event datasets) into event text files. In addition, we also provide scripts to convert a folder containing image reconstructions back to a rosbag (or to append image reconstructions to an existing rosbag).

Note: these scripts need ROS installed and sourced. They are not part of the uv environment.

rosbag -> events.txt

To extract the events from a rosbag to a zip file containing the event data:

python scripts/extract_events_from_rosbag.py /path/to/rosbag.bag \
  --output_folder=/path/to/output/folder \
  --event_topic=/dvs/events

image reconstruction folder -> rosbag

python scripts/image_folder_to_rosbag.py \
  --datasets dynamic_6dof \
  --image_folder /path/to/image/folder \
  --output_folder /path/to/output_folder \
  --image_topic /dvs/image_reconstructed

Append image_reconstruction_folder to an existing rosbag

cd scripts
python embed_reconstructed_images_in_rosbag.py \
  --rosbag_folder /path/to/rosbag/folder \
  --datasets dynamic_6dof \
  --image_folder /path/to/image/folder \
  --output_folder /path/to/output_folder \
  --image_topic /dvs/image_reconstructed

Generating a video reconstruction (with a fixed framerate)

It can be convenient to convert an image folder to a video with a fixed framerate (for example for use in a video editing tool). You can proceed as follows:

export FRAMERATE=30
python resample_reconstructions.py -i /path/to/input_folder -o /tmp/resampled -r $FRAMERATE
ffmpeg -framerate $FRAMERATE -i /tmp/resampled/frame_%010d.png video_"$FRAMERATE"Hz.mp4

Acknowledgements

This code borrows from the following open source projects, whom we would like to thank: