Super-Resolution.Benckmark

March 31, 2018 · View on GitHub

A curated list of super-resolution resources and a benchmark for single image super-resolution algorithms.

See my implementated super-resolution algorithms:

TODO

Build a benckmark like SelfExSR_Code

State-of-the-art algorithms

Classical Sparse Coding Method

  • ScSR [Web]
  • Image super-resolution as sparse representation of raw image patches (CVPR2008), Jianchao Yang et al.
  • Image super-resolution via sparse representation (TIP2010), Jianchao Yang et al.
  • Coupled dictionary training for image super-resolution (TIP2011), Jianchao Yang et al.

Anchored Neighborhood Regression Method

  • ANR [Web]
  • Anchored Neighborhood Regression for Fast Example-Based Super-Resolution (ICCV2013), Radu Timofte et al.
  • A+ [Web]
  • A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution (ACCV2014), Radu Timofte et al.
  • IA [Web]
  • Seven ways to improve example-based single image super resolution (CVPR2016), Radu Timofte et al.

Self-Exemplars

  • SelfExSR [Web]
  • Single Image Super-Resolution from Transformed Self-Exemplars (CVPR2015), Jia-Bin Huang et al.

Bayes

  • NBSRF [Web]
  • Naive Bayes Super-Resolution Forest (ICCV2015), Jordi Salvador et al.

Deep Learning Method

  • SRCNN [Web] [waifu2x by nagadomi]
  • Image Super-Resolution Using Deep Convolutional Networks (ECCV2014), Chao Dong et al.
  • Image Super-Resolution Using Deep Convolutional Networks (TPAMI2015), Chao Dong et al.
  • CSCN [Web]
  • Deep Networks for Image Super-Resolution with Sparse Prior (ICCV2015), Zhaowen Wang et al.
  • Robust Single Image Super-Resolution via Deep Networks with Sparse Prior (TIP2016), Ding Liu et al.
  • VDSR [Web] [Unofficial Implementation in Caffe]
  • Accurate Image Super-Resolution Using Very Deep Convolutional Networks (CVPR2016), Jiwon Kim et al.
  • DRCN [Web]
  • Deeply-Recursive Convolutional Network for Image Super-Resolution (CVPR2016), Jiwon Kim et al.
  • ESPCN [PDF]
  • Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network (CVPR2016), Wenzhe Shi et al.
  • Is the deconvolution layer the same as a convolutional layer? [PDF]
  • Checkerboard artifact free sub-pixel convolution [PDF]
  • FSRCNN [Web]
  • Acclerating the Super-Resolution Convolutional Neural Network (ECCV2016), Dong Chao et al.
  • LapSRN [Web]
  • Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution (CVPR 2017), Wei-Sheng Lai et al.
  • EDSR [PDF]
  • Enhanced Deep Residual Networks for Single Image Super-Resolution (Winner of NTIRE2017 Super-Resolution Challenge), Bee Lim et al.

Perceptual Loss and GAN

  • Perceptual Loss [PDF]
  • Perceptual Losses for Real-Time Style Transfer and Super-Resolution (ECCV2016), Justin Johnson et al.
  • SRGAN [PDF]
  • Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (CVPR2017), Christian Ledig et al.
  • AffGAN [PDF]
  • AMORTISED MAP INFERENCE FOR IMAGE SUPER-RESOLUTION (ICLR2017), Casper Kaae Sønderby et al.
  • EnhanceNet [PDF]
  • EnhanceNet: Single Image Super-Resolution through Automated Texture Synthesis, Mehdi S. M. Sajjadi et al.
  • neural-enchance [Github]

Video SR

  • VESPCN [PDF]
  • Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation (CVPR2017), Jose Caballero et al.

Dicussion

Deconvolution and Sub-Pixel Convolution

Datasets

Test DatasetImage source
Set 5Bevilacqua et al. BMVC 2012
Set 14Zeyde et al. LNCS 2010
BSD 100Martin et al. ICCV 2001
Urban 100Huang et al. CVPR 2015
Train DatasetImage source
Yang 91Yang et al. CVPR 2008
BSD 200Martin et al. ICCV 2001
General 100Dong et al. ECCV 2016
ImageNetOlga Russakovsky et al. IJCV 2015
COCOTsung-Yi Lin et al. ECCV 2014

Quantitative comparisons

Results from papers of VDSR, DRCN, CSCN and IA.

Note: IA use enchanced prediction trick to improve result.

Results on Set 5
ScaleBicubicA+SRCNNSelfExSRCSCNVDSRDRCNIA
2x - PSNR/SSIM33.66/0.992936.54/0.954436.66/0.954236.49/0.953736.93/0.955237.53/0.958737.63/0.958837.39/
3x - PSNR/SSIM30.39/0.868232.59/0.908832.75/0.909032.58/0.909333.10/0.914433.66/0.921333.82/0.922633.46/
4x - PSNR/SSIM28.42/0.810430.28/0.860330.48/0.862830.31/0.861930.86/0.873231.35/0.883831.53/0.885431.10/
Results on Set 14
ScaleBicubicA+SRCNNSelfExSRCSCNVDSRDRCNIA
2x - PSNR/SSIM30.24/0.868832.28/0.905632.42/0.906332.22/0.903432.56/0.907433.03/0.912433.04/0.911832.87/
3x - PSNR/SSIM27.55/0.774229.13/0.818829.28/0.820929.16/0.819629.41/0.823829.77/0.831429.76/0.831129.69/
4x - PSNR/SSIM26.00/0.702727.32/0.749127.49/0.750327.40/0.751827.64/0.758728.01/0.767428.02/0.767027.88/
Results on BSD 100
ScaleBicubicA+SRCNNSelfExSRCSCNVDSRDRCNIA
2x - PSNR/SSIM29.56/0.843131.21/0.886331.36/0.887931.18/0.885531.40/0.888431.90/0.896031.85/0.894231.79/
3x - PSNR/SSIM27.21/0.738528.29/0.783528.41/0.786328.29/0.784028.50/0.788528.82/0.797628.80/0.796328.76/
4x - PSNR/SSIM25.96/0.667526.82/0.708726.90/0.710126.84/0.710627.03/0.716127.29/0.725127.23/0.723327.25/