Need help with SelfExSR?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

jbhuang0604
489 Stars 255 Forks 29 Commits 5 Opened issues

Description

Single Image Super-Resolution from Transformed Self-Exemplars (CVPR 2015)

Services available

!
?

Need anything else?

Contributors list

# 42,074
Compute...
MATLAB
Python
Shell
28 commits

Single Image Super-Resolution from Transformed Self-Exemplars (CVPR 2015)

Introduction

This is the research code for the paper:

Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja, "Single Image Super-Resolution from Transformed Self-Exemplars", CVPR 2015 PDF

The proposed algorithm achieves the state-of-the-art performance on image super-resolution without the need of any external training dataset, feature extraction and complicated learning algorithms. For more details, please visit our Project page.

All the datasets (Set5, Set14, Urban 100, BSD 100, Sun-Hays 80), precomputed results and visual comparisons can be found in the following sections.

Citation

If you find the code and dataset useful in your research, please consider citing:

@inproceedings{Huang-CVPR-2015,
    title={Single Image Super-Resolution From Transformed Self-Exemplars},
    Author = {Huang, Jia-Bin and Singh, Abhishek and Ahuja, Narendra},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
    pages={5197--5206},
    Year = {2015}
}

Contents

| Folder | description | | ---|---| |cache | cached data for vanishing point detection| |data|Testing images of five datasets (Set5, Set14, Urban 100, BSD 100, Sun-Hays 80). All the images have been cropped according to the desired super-resolution factor. This avoids misalignment of the groundtruth high-resolution images and the super-resolved images| |external|We use the vgginterp2 from

imrender
to perform bilinear interpolation| |quanteval|Quantitative evaluation code| |reference| A copy of the CVPR paper and the bibtex| |source|MATLAB source code|

To run the algorithm on all datasets, simply run the

sr_demo_bacth.m
. Note that it is an educational code that is not optimized for speed. If timing is a concern, you can achieve visually similar results with small numbers of iterations, e.g., set the number of iterations
opt.numIter = 5;
in the file
sr_init_opt.m
. An example of the speed and quality trade-off can be found in Fig. 10 in the paper.

Feedbacks and comments are welcome! Feel free to contact me via [email protected]

Enjoy!

Note: For vanishing point detection only windows executable is provided (from Image Completion using Planar Structure Guidance), a cross-platform version will be included later.

Comparison with the state-of-the-art

Datasets

The full super-resolution results on Set 5, Set 14, Urban 100, BSD 100 and Sun-Hays 80 are available.

| Dataset | Image source | Download full results | |---- | ---|----| ----| | Set 5 | Bevilacqua et al. BMVC 2012 | link (16.1 MB)| | Set 14 | Zeyde et al. LNCS 2010 | link (86.0 MB)| | Urban 100 | Huang et al. CVPR 2015 | link (1.14 GB)| | BSD 100 | Martin et al. ICCV 2001 | link (568 MB)| | Sun-Hays 80 | Sun and Hays ICCP 2012 | link (311 MB)|

Set 5 dataset - link Set 5

Set 14 dataset - link Set 14

Urban 100 dataset - link Urban 100

BSD 100 dataset - link BSD 100

Sun-Hays 80 dataset - link Sun-Hays 80

State-of-the-art image super-resolution algorithms

In each dataset, we include results of the state-of-the-art single image super-resolution algorithms:

| Image | Description| | ---|---| | HR| High-resolution images. All images were cropped so that each dimension is a multiplication of the super-resolution factor. This avoids the misalignment problem in the quantitative comparison. | | LR | Low-resolution test images generated with bicubic kernel downsampling.| |bicubic|Bicubic interpolation| |nearest|Nearest-neighbor interpolation| |SelfExSR|Our result| |A+|R. Timofte, V. De Smet, and L. Van Gool, A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. In Asian Conference on Computer Vision (ACCV 2014). Code available here| |Abhishek|A. Singh and N. Ahuja, Super-Resolution Using Sub-Band Self-Similarity. In Asian Conference on Computer Vision (ACCV 2014). No publicly implementation available. Results were provided by the authors.| |Kim|K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 1127-1133, 2010. Code available here| |Glasner| Daniel Glasner, Shai Bagon, Michal Irani. Super-Resolution From a Single Image, In International Conference on Computer Vision (ICCV 2009). No public implementation available. Results were generated by our own implementation.| |ScSR|Jianchao Yang, John Wright, Thomas Huang, and Yi Ma. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, Vol 19, Issue 11, pp2861-2873, 2010. Code available here| |SRCNN| Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang. Learning a Deep Convolutional Network for Image Super-Resolution, in European Conference on Computer Vision (ECCV 2014). Code available here|

Qualitative comparison

In our supplementary material, we includde 120 sample comparisons with the state-of-the-art algorithms. Download the document here.

You can browse and compare our results with other methods via the following links. * Urbana 100 - Super-resolution factor 4x * BSD 100 - Super-resolution factor 3x * Sun-Hays 80 - Super-resolution factor 8x

Quantitative comparisons

We report three types of metrics * PSNR: Peak signal-to-noise ratio * SSIM: Structural similarity index * IFC: Information fidelity criterion

Results on Set 5

| Scale | Bicubic | ScSR | Kim | Sub-band | Glasner |SRCNN | A+ | Ours | |:---------:|:-------:|:--------:|:------:|:------------:|:---------:|:--------:|:------:|:----:| | 2x - PSNR| 33.64 | 35.78 | 36.24 | Sub-band | 35.43 | 36.28 | A+ | 36.50 | | 3x - PSNR| 30.39 | 31.34 | 32.30 | Sub-band | 31.10 | 32.37 | A+ | 32.62 | | 4x - PSNR| 28.42 | 29.07 | 30.07 | Sub-band | 28.84 | 30.08 | A+ | 30.33 | || | 2x - SSIM| 0.9292 | 0.9485 | 0.9518 | Sub-band | 0.9452 | 0.9509 | A+ | 0.9537 | | 3x - SSIM| 0.8678 | 0.8869 | 0.9041 | Sub-band | 0.8811 | 0.9025 | A+ | 0.9094 | | 4x - SSIM| 0.8101 | 0.8263 | 0.8553 | Sub-band | 0.8210 | 0.8525 | A+ | 0.8623 | || | 2x - IFC| 5.72 | 6.94 | 7.05 | Sub-band | 6.70 | 6.85 | A+ | 7.83 | | 3x - IFC| 3.45 | 3.98 | 4.25 | Sub-band | 3.68 | 4.11 | A+ | 4.76 | | 4x - IFC| 2.28 | 2.57 | 2.82 | Sub-band | 2.42 | 2.76 | A+ | 3.19 |

Results on Set 14

| Scale | Bicubic | ScSR | Kim | Sub-band | Glasner |SRCNN | A+ | Ours | |:---------:|:-------:|:--------:|:------:|:------------:|:---------:|:--------:|:------:|:----:| | 2x - PSNR| 30.22 | 31.64 | 32.14 | Sub-band | 31.41 | 32.00 | A+ | 32.23 | | 3x - PSNR| 27.53 | 28.19 | 28.96 | Sub-band | 28.21 | 28.90 | A+ | 29.16 | | 4x - PSNR| 25.99 | 26.40 | 27.18 | Sub-band | 26.43 | 27.13 | A+ | 27.40 | || | 2x - SSIM| 0.8683 | 0.8940 | 0.9031 | Sub-band | 0.8881 | 0.9012 | A+ | 0.9036 | | 3x - SSIM| 0.7737 | 0.7977 | 0.8140 | Sub-band | 0.7926 | 0.8124 | A+ | 0.8197 | | 4x - SSIM| 0.7023 | 0.7218 | 0.7434 | Sub-band | 0.7163 | 0.7395 | A+ | 0.7518 | || | 2x - IFC| 5.74 | 6.83 | 6.92 | Sub-band | 6.47 | 6.68 | A+ | 7.60 | | 3x - IFC| 3.33 | 3.75 | 3.92 | Sub-band | 3.59 | 3.81 | A+ | 4.38 | | 4x - IFC| 2.18 | 2.46 | 2.57 | Sub-band | 2.30 | 2.50 | A+ | 2.90 |

Results on Urban 100

| Scale | Bicubic | ScSR | Kim | Sub-band | Glasner |SRCNN | A+ | Ours | |:---------:|:-------:|:--------:|:------:|:------------:|:---------:|:--------:|:------:|:----:| | 2x - PSNR| 26.66 | 28.26 | 28.74 | 28.34 | 27.85 | 28.65 | 28.87 | 29.38 | | 4x - PSNR| 23.14 | 24.02 | 24.20 | 24.19 | 23.58 | 24.14 | 24.34 | 24.82 | || | 2x - SSIM| 0.8408 | 0.8828 | 0.8940 | 0.8820 | 0.8709 | 0.8909 | 0.8957 | 0.9032 | | 4x - SSIM| 0.6573 | 0.7024 | 0.7104 | 0.7115 | 0.6736 | 0.7047 | 0.7195 | 0.7386 | || | 2x - IFC| 5.72 | 6.98 | 6.86 | 7.08 | 6.17 | 6.66 | 8.02 | 7.96 | | 4x - IFC| 2.27 | 2.75 | 2.71 | 2.72 | 2.35 | 2.63 | 3.16 | 3.33 |

Results on BSD 100

| Scale | Bicubic | ScSR | Kim | Sub-band | Glasner |SRCNN | A+ | Ours | |:---------:|:-------:|:--------:|:------:|:------------:|:---------:|:--------:|:------:|:----:| | 2x - PSNR| 29.55 | 30.77 | 31.11 | 30.73 | 30.28 | 31.11 | 31.22 | 31.18 | | 3x - PSNR| 27.20 | 27.72 | 28.17 | 27.88 | 27.06 | 28.20 | 28.30 | 28.30 | | 4x - PSNR| 25.96 | 26.61 | 26.71 | 26.60 | 26.17 | 26.70 | 26.82 | 26.85 | || | 2x - SSIM| 0.8425 | 0.8744 | 0.8840 | 0.8774 | 0.8621 | 0.8835 | 0.8862 | 0.8855 | | 3x - SSIM| 0.7382 | 0.7647 | 0.7788 | 0.7714 | 0.7368 | 0.7794 | 0.7836 | 0.7843 | | 4x - SSIM| 0.6672 | 0.6983 | 0.7027 | 0.7021 | 0.6747 | 0.7018 | 0.7089 | 0.7108 | || | 2x - IFC| 5.26 | 6.20 | 6.30 | 6.36 | 5.56 | 6.09 | 7.15 | 6.84 | | 3x - IFC| 3.00 | 3.37 | 3.49 | 3.17 | 2.72 | 3.39 | 3.92 | 3.81 | | 4x - IFC| 1.91 | 2.22 | 2.20 | 2.18 | 1.86 | 2.18 | 2.51 | 2.46 |

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.