ELEDNet (ECCV 2024)

Official repository for the ECCV 2024 paper, "Towards Real-world Event-guided Low-light Video Enhancement and Deblurring."

[Paper] [Supp]

Video Demos

Downloading the RELED datasets

Please download and unzip the RELED dataset.

• [RELED-Train] / [RELED-Test]

The dataset follows the below directory format:

├── RELED/
    ├── train/
    │   ├── 0000/
    │   │   ├── blur_processed/
    │   │   │   ├── 00000.png
    │   │   │   ├── ...
    │   │   │   └── 00148.png
    │   │   ├── gt_processed/
    │   │   │   ├── 00000.png
    │   │   │   ├── ...
    │   │   │   └── 00148.png
    │   │   ├── events/
    │   │   │   ├── 00000.npz
    │   │   │   ├── ...
    │   │   │   └── 00148.npz
    │   │   └── event_voxel/
    │   │       ├── 00000.npz
    │   │       ├── ...
    │   │       └── 00148.npz
    │   ├── 0001/
    │   │   ├── ...
    ├── test/
    │   ├── 0000/
    │   │   ├── ...
    │   ├── 0001/
    │   │   ├── ...

Sub-directory Descriptions:

• blur_processed: Contains low-light blurred images (*.png files).
• gt_processed: Contains normal-light sharp images (*.png files).
• events: Contains raw event data in .npz format.
• event_voxel: Contains event voxel data in .npz format.

Reading Raw Event Data (events) and Event Voxel Data(event_voxel):

To read event and event voxel data from .npz files using Python and NumPy:

import numpy as np

# Replace YOUR_EVENT_DIR with the path to the directory containing the .npz files for events
event_data = np.load('YOUR_EVENT_DIR/*.npz')['data']

Requirements

• PyTorch 1.9
• CUDA 11.2
• python 3.7

Quick Train model

Download repository:

$ git clone https://github.com/intelpro/ELEDNet

If you want to start training our model, you need to preprocess the raw dataset first.

Run the following command to preprocess the dataset:

$ python utils/make_train_dataset --train_data_dir TRAIN_DATASET_DIR

• --train_data_dir TRAIN_DATASET_DIR: Specifies the directory containing the training dataset of the RELED dataset.
• The process divides the blur, event voxel, and ground truth (GT) data into four parts to enhance training speed.

Once preprocessing is complete, you can proceed to the model training step.

$ python train.py --data_dir DATSET_DIR

• --data_dir DATSET_DIR: Specifies the directory containing the complete RELED dataset, including both training and test sets.

Quick Test model

Download repository:

$ git clone https://github.com/intelpro/ELEDNet

Download the network weights (trained on the RELED dataset) and place the downloaded model inside the ./pretrained_model directory.

🔗 Download Ours Weights

# Ensure the directory exists
mkdir -p pretrained_model

# Move the downloaded model to the correct location
mv /path/to/downloaded/Ours_RELED.pth ./pretrained_model/

Generate output images using our model and sample data provided in this repository.

$ python test_sample.py --resume_ckpt True --ckpt_dir ./pretrained_model/Ours_RELED.pth

Test model

If you want to test full RELED dataset, please generate output images using following command

$ python test_model.py --data_dir RELED_PATH --resume_ckpt True --ckpt_dir PATH_CKPT --saved_dir SAVED_DIR

• --data_dir PATH_RELED : Path to the RELED dataset
• --resume_ckpt True : Enables loading of a pretrained model checkpoint.
• --ckpt_dir PATH_CKPT : Path to the pretrained checkpoint file.
• --saved_dir SAVED_DIR : Directory where output images will be saved.

Reference

Taewoo Kim, Jaeseok Jeong, Hoonhee Cho, Yuhwan Jeong, and Kuk-Jin Yoon, "Towards Real-World Event-guided Low-Light Video Enhancement and Deblurring," In ECCV, 2024.

@inproceedings{kim2024towards,
  title={Towards Real-world Event-guided Low-light Video Enhancement and Deblurring},
  author={Kim, Taewoo and Jeong, Jaeseok and Cho, Hoonhee and Jeong, Yuhwan and Yoon, Kuk-Jin},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  pages={433--451},
  year={2024},
  publisher={Springer}
}

Contact

If you have any question, please send an email to taewoo(an625148@gmail.com)

License

The project codes and datasets can be used for research and education only.