Need help with Video-and-Single-Image-Deraining?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

hongwang01
141 Stars 42 Forks 14 Commits 1 Opened issues

Description

categorize current rain removal methods and fairly evaluate their deraining ability based on diverse benchmark datasets

Services available

!
?

Need anything else?

Contributors list

No Data

A Survey on Rain Removal from Video and Single Image

Hong Wang, Yichen Wu, Minghan Li, Qian Zhao, and Deyu Meng

[Arxiv]

Citation

@article{WangA,
  title={A Survey on Rain Removal from Video and Single Image}, 
  author={Wang, Hong and Wu, Yichen and Li, Minghan and Zhao, Qian and Meng, Deyu}, 
  journal={arXiv preprint arXiv:1909.08326},
  year={2019}
}

Physical Properties of Raindrops

  • Gemometric Property
    • Terminal velocity of raindrops aloft (JAMC1969), Foote et al [PDF]
    • A new model for the equilibrium shape of raindrops (JAS1987), Beard et al. [PDF]
  • Brightness Property
    • Photometric model of a rain drop (Technical Report, Columbia University2004), Garg et al [PDF]
    • Vision and Rain (IJCV2007), Garg et al [Project][PDF]
  • Chromatic Property
    • Rain removal in video by combining temporal and chromatic properties (ICME2006), Zhang et al [Project][PDF]
  • Spatial and Temporal Propety
    • Simulation of rain in videos (TAS2003), Starik et al [PDF]
    • Pixel based temporal analysis using chromatic property for removing rain from videos (CIC2009), Liu et al [PDF] ## Video Deraining Methods
  • Time Domain
    • Detection and removal of rain from videos (CVPR2004), Garg et al [Project][PDF]
    • When does camera see rain? (ICCV2005), Garg et al [Project][PDF]
    • Rain removal using kalman filter in video (ICSMA2008), Park et al [PDF]
    • Using the shape characteristics of rain to identify and remove rain from video (S+SSPR2008), Brewer et al [PDF]
    • The application of histogram on rain detection in video (JCIS2008), Zhao et al [PDF]
    • Rain or snow detection in image sequences through use of a histogram of orientation of streaks (IJCV2011), Bossu et al [PDF]
    • A probabilistic approach for detection and removal of rain from videos (IETE JR2011), Tripathi et al [PDF]
    • Video post processing: low latency spatiotemporal approach for detection and removal of rain (IET IP2012), Tripathi et al [PDF]
    • Removal of rain from videos: a review (SIVP2014), Tripathi et al [PDF]
    • Stereo video deraining and desnowing based on spatiotemporal frame warping (ICIP2014), Kim et al [PDF]
  • Frequency Domain

    • Spatio-temporal frequency analysis for removing rain and snow from videos (PACV2007), Barnum et al [Project] [PDF]
    • Analysis of rain and snow in frequency space (IJCV2010), Barnum et al [Project] [PDF]
  • Low Rank and Sparsity

    • A generalized low-rank appearance model for spatio-temporally correlated rain streaks (ICCV2013), Chen et al [PDF]
    • A rain pixel recovery algorithm for videos with highly dynamic scenes (TIP2013), Chen et al [PDF]
    • Video deraining and desnowing using temporal correlation and low-rank matrix completion (TIP2015), Kim et al [PDF] [Code]
    • Adherent raindrop modeling, detection and removal in video (TPAMI2016), You et al. [Project] [PDF]
    • Video desnowing and deraining based on matrix decomposition (CVPR2017), Ren et al [PDF] [Code]
    • A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors (CVPR2017), Jiang et al [PDF]
    • Should We encode rain streaks in video as deterministic or stochastic? (ICCV2017), Wei et al [PDF] [Code]
    • A directional global sparse model for single image rain removal (AMM2018), Deng et al [PDF] [Code]
    • Video rain streak removal by multiscale convolutional sparse coding (CVPR2018), Li et al [Project] [PDF] [Code]
    • Fastderain: A novel video rain streak removal method using directional gradient priors (TIP2019), Jiang et al [PDF] [Code]
  • Deep Learning

    • Robust video content alignment and compensation for rain removal in a cnn framework (CVPR2018), Chen et al [PDF] [Code]
    • Erase or fill? deep joint recurrent rain removal and reconstruction in videos (CVPR2018), Liu et al. [Project][PDF] [Code]
    • D3R-Net: dynamic routing residue recurrent network for video rain removal (TIP2018), Liu et al. [PDF]
    • Frame consistent recurrent video deraining with dual-level flow (CVPR2019), Yang et al. [Code]
    • Self-Learning Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence(CVPR2020), Yang et al.[PDF][Supplementray Materials] [Code]
  • Reivew paper

    • Removal of rain from videos: a review (SIVP2014), Tripathi et al [PDF]
    • A Survey on Rain Removal from Video and Single Image (Arxiv2019), Wang et al. [PDF] [Code]

Single Image Deraining Methods

  • Filter based methods

    • Guided image filtering (ECCV2010), He et al. [Project] [PDF] [Code]
    • Removing rain and snow in a single image using guided filter (CSAE2012), Xu et al. [PDF]
    • An improved guidance image based method to remove rain and snow in a single image (CIS2012), Xu et al. [PDF]
    • Single-image deraining using an adaptive nonlocal means filter (ICIP2013), Kim et al. [PDF]
    • Single-image-based rain and snow removal using multi-guided filter (NIPS2013), Zheng et al. [PDF]
    • Single image rain and snow removal via guided L0 smoothing filter (Multimedia Tools and Application2016), Ding et al. [PDF]
  • Prior based methods

    • Automatic single-image-based rain streaks removal via image decomposition (TIP2012), Kang et al [PDF] [Code]
    • Self-learning-based rain streak removal for image/video (ISCS2012), Kang et al. [PDF]
    • Single-frame-based rain removal via image decomposition (ICA2013), Fu et al. [PDF]
    • Exploiting image structural similarity for single image rain removal (ICIP2014), Sun et al. [PDF]
    • Visual depth guided color image rain streaks removal using sparse coding (TCSVT2014), Chen et al [PDF]
    • Removing rain from a single image via discriminative sparse coding (ICCV2015), Luo et al [PDF] [Code] pwd: d229
    • Rain streak removal using layer priors (CVPR2016), Li et al [PDF] [Code]
    • Single image rain streak decomposition using layer priors (TIP2017), Li et al [PDF]
    • Error-optimized dparse representation for single image rain removal (IEEE TIE2017), Chen et al [PDF]
    • A hierarchical approach for rain or snow removing in a single color image (TIP2017), Wang et al. [PDF]
    • Joint bi-layer optimization for single-image rain streak removal (ICCV2017), Zhu et al. [PDF]
    • Convolutional sparse and low-rank codingbased rain streak removal (WCACV2017), Zhang et al [PDF]
    • Joint convolutional analysis and synthesis sparse representation for single image layer separation (CVPR2017), Gu et al [PDF] [Code]
    • Single image deraining via decorrelating the rain streaks and background scene in gradient domain (PR2018), Du et al [PDF]
  • Deep Learning

    • Restoring an image taken through a window covered with dirt or rain (ICCV2013), Eigen et al. [Project] [PDF] [Code]
    • Attentive generative adversarial network for raindrop removal from a single image (CVPR2018), Qian et al [Project] [PDF]
    • Clearing the skies: A deep network architecture for single-image rain streaks removal (TIP2017), Fu et al. [Project] [PDF] [Code]
    • Removing rain from single images via a deep detail network (CVPR2017), Fu et al. [Project] [PDF] [Code]
    • Image de-raining using a conditional generative adversarial network (Arxiv2017), Zhang et al [PDF] [Code]
    • Deep joint rain detection and removal from a single image (CVPR2017), Yang et al.[Project] [PDF] [Code]
    • Residual guide feature fusion network for single image deraining (ACMMM2018), Fan et al. [Project] [PDF]
    • Fast single image rain removal via a deep decomposition-composition network (Arxiv2018), Li et al [Project]) [PDF] [Code]
    • Density-aware single image de-raining using a multi-stream dense network (CVPR2018), Zhang et al [PDF] [Code]
    • Recurrent squeeze-and-excitation context aggregation net for single image deraining (ECCV2018), Li et al. [PDF] [Code]
    • Rain streak removal for single image via kernel guided cnn (Arxiv2018), Wang et al [PDF]
    • Physics-based generative adversarial models for image restoration and beyond (Arxiv2018), Pan et al [PDF]
    • Learning dual convolutional neural networks for low-level vision (CVPR2018), Pan et al [Project] [PDF] [Code]
    • Non-locally enhanced encoder-decoder network for single image de-raining (ACMMM2018), Li et al [PDF] [Code]
    • Single image rain removal via a deep decomposition-composition network (CVIU2019), Li et al.
    • Unsupervised single image deraining with self-supervised constraints (ICIP2019), Jin et al [PDF]
    • Residual multiscale based single image deraining (BMVC2019), Zheng et al.
    • Erl-net: Entangled representation learning for single image de-raining (ICCV2019), Wang et al. [code]
    • Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining (CVPR2019), Rajeev Yasarla et al.[Code]
    • Heavy rain image restoration: Integrating physics model and conditional adversarial learning (CVPR2019), Li et al.[Code]
    • Progressive image deraining networks: A better and simpler baseline (CVPR2019), Ren et al [PDF] [Code]
    • Spatial attentive single-image deraining with a high quality real rain dataset (CVPR2019), Wang et al [Project] [PDF] [Code]
    • Lightweight pyramid networks for image deraining (TNNLS2019), Fu et al [PDF] [Code]
    • Joint rain detection and removal from a single image with contextualized deep networks (TPAMI2019), Yang et al [PDF] [Code]
    • Scale-free single image deraining via visibility-enhanced recurrent wavelet learning (TIP2019), Yang et al.[PDF]
    • Towards scale-free rain streak removal via selfsupervised fractal band learning (AAAI2020), Yang et al.[Code]
    • Structural Residual Learning for Single Image Rain Removal(Arxiv2020), Wang et al. [PDF]
    • All in One Bad Weather Removal Using Architectural Search (CVPR2020), Li et al.[PDF]
    • Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes(CVPR2020), Rajeev Yasarla et al. [Code]
    • Multi-Scale Progressive Fusion Network for Single Image Deraining(CVPR2020), Jiang et al. [Code]
    • Detail-recovery Image Deraining via Context Aggregation Networks(CVPR2020), Deng et al.[Code]
    • Variational image deraining(WACV2020), Du et al.[PDF]
  • Joint Model-driven and Data-driven

    • Deep Layer Prior Optimization for Single Image Rain Streaks Removal (ICASSP2018), Liu et al [PDF]
    • Learning bilevel layer priors for single image rain streaks removal (SP Letters 2018), Mu et al [PDF]
    • Semi-supervised transfer learning for image rain removal (CVPR2019), Wei et al [PDF] [Code]
    • Knowledge-driven deep unrolling for robust image layer separation (TNNLS2019), Liu et al.
    • A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020), Wang et al [PDF][Supplementary Materials [Code]
  • Reivew paper

    • Single image deraining: A comprehensive benchmark analysis(CVPR2019), Li et al.[PDF] [Code]
    • A Survey on Rain Removal from Video and Single Image (Arxiv2019), Wang et al. [PDF] [Code]
    • Single image deraining: From model-based to data-driven and beyond(TPAMI2020), Yang et al.[Code]

Datasets and Discriptions

*We note that:

i. *RainTrainL/Rain100L** and RainTrainH/Rain100H are synthesized by Yang Wenhan. Rain12600/Rain1400 is from Fu Xueyang and Rain12 is from Li Yu.*

ii. In video experiment, the rain-removed results of the deep learning method are provided by the author Yang Wenhan. Really thanks!

iii. In single image experiment, we seperately retrain all the recent state-of-the-art methods via the three training datasets: *RainTrainL(200 input/clean image pairs), **RainTrainH(1800 pairs), and Rain12600(12600 pairs), and then evaluate their rain removal performance based on the correponding test datasets: Rain100L(100 pairs), Rain100H(100 pairs), and Rain1400(1400 pairs). Besides, the trained model obtained by RainTrainL is adpoted to predict rain-removed results of Rain12(12 pairs). Moreover, we utilize the Internet-Data(147 input images) and SPA-Data(1000 pairs) to compare the generalization ability.*

iiii. In single image experiment, when training the semi-supervised method--SIRR, we always utilize *Internet-Data** as unsupervised samples*.

Image Quality Metrics

*Please note that all quantitative results in our survey paper are computed based on Y channel.

Contact

If you have any question, please feel free to concat Hong Wang (Email: [email protected]).

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.