By Pontus Ebelin and Tomas Akenine-Möller, with Jim Nilsson, Magnus Oskarsson, Kalle Åström, Mark D. Fairchild, and Peter Shirley.

This repository holds implementations of the LDR-FLIP and HDR-FLIP image error metrics as loss modules in PyTorch.

License

Copyright © 2020-2024, NVIDIA Corporation & Affiliates. All rights reserved.

This work is made available under a BSD 3-Clause License.

The repository distributes code for tinyexr, which is subject to a BSD 3-Clause License,
and stb_image, which is subject to an MIT License.

For individual contributions to the project, please confer the Individual Contributor License Agreement.

For business inquiries, please visit our website and submit the form: NVIDIA Research Licensing.

PyTorch (Loss Function)

• Setup (with Anaconda3):

  conda create -n flip_dl python numpy matplotlib
  conda activate flip_dl
  conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
  conda install -c conda-forge openexr-python
  

• Remember to activate the flip_dl environment through conda activate flip_dl before using the loss function.
• LDR- and HDR-FLIP are implemented as loss modules in flip_evaluator/pytorch/flip_loss.py. An example where the loss function is used to train a simple autoencoder is provided in flip_evaluator/pytorch/train.py.
• Tested on Windows with Conda 4.10.0, CUDA 11.2, Python 3.9.4, PyTorch 1.8.1, NumPy 1.20.1, and OpenEXR b1.3.2.
• Per default, the loss function returns the mean of the error maps. To return the full error maps, remove torch.mean() from the forward() function.
• For LDR-FLIP, the images are assumed to be in sRGB space (change the color space transform in LDRFLIPLoss's forward() function to linrgb2ycxcz if your network's output is in linear RGB), in the [0,1] range.
• Both LDR- and HDR-FLIP takes an optional argument describing the assumed number of pixels per degree of the observer. Per default, it is assume that the images are viewed at a distance 0.7 m from a 0.7 m wide 4K monitor.
• The HDRFLIPLoss can take three additional, optional arguments: tone_mapper, start_exposure, and stop_exposure. tone_mapper is a string describing the tone mapper that HDR-FLIP should assume, for which the choices are aces (default), hable, and reinhard. The default assumption is the ACES tone mapper. start_exposure, and stop_exposure should have Nx1x1x1 layout and hold the start and stop exposures, respectively, used for each of the N reference/test pairs in the batch. Per default, HDRFLIPLoss computes start and stop exposures as described in the paper. NOTE: When start and/or stop exposures are not provided, HDR-FLIP is not symmetric. The user should therefore make sure to input the test images as the first argument and the reference image as the second argument to the HDRFLIPLoss's forward() function.
• flip_evaluator/tests/test_pytorch.py contains simple tests used to test whether code updates alter results and flip_evaluator/pytorch/data.py contains image loading/saving functions.