Journal of Computer Science and Technology ›› 2023, Vol. 38 ›› Issue (2): 439-454.doi: 10.1007/s11390-022-0979-2

Special Issue: Artificial Intelligence and Pattern Recognition

• Regular Paper • Previous Articles     Next Articles

Single Image Deraining Using Residual Channel Attention Networks

Di Wang (王　迪), Jin-Shan Pan* (潘金山), Member, IEEE, and Jin-Hui Tang (唐金辉), Distinguish Member, CCF, Senior Member, IEEE, Member, ACM   

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
  • Received:2020-09-09 Revised:2022-09-07 Accepted:2022-09-23 Online:2023-05-10 Published:2022-09-24
  • Contact: Jin-Shan Pan E-mail:jspan@njust.edu.cn
  • About author:Jin-Shan Pan is a professor at the School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing. He received his Ph.D. degree in computational mathematics from the Dalian University of Technology, Dalian, in 2017. His research interests include deblurring, image/video analysis and enhancement, and related vision problems. He is a member of IEEE.
  • Supported by:
    This work was supported by the National Key Research and Development Program of China under Grant No. 2018AAA0102001 and the Fundamental Research Funds for the Central Universities of China under Grant No. 30920041109.

PDF

83

Supplement

1

Image deraining is a highly ill-posed problem. Although significant progress has been made due to the use of deep convolutional neural networks, this problem still remains challenging, especially for the details restoration and generalization to real rain images. In this paper, we propose a deep residual channel attention network (DeRCAN) for deraining. The channel attention mechanism is able to capture the inherent properties of the feature space and thus facilitates more accurate estimations of structures and details for image deraining. In addition, we further propose an unsupervised learning approach to better solve real rain images based on the proposed network. Extensive qualitative and quantitative evaluation results on both synthetic and real-world images demonstrate that the proposed DeRCAN performs favorably against state-of-the-art methods.

Key words: deraining; deep convolutional neural network (DCNN); channel attention; detail restoration; unsupervised finetuning;

[1] Zhang D, Zhang H, Tang J, Wang M, Hua X, Sun Q. Feature pyramid transformer. In Proc. the 16th European Conference on Computer Vision, August 2020, pp.323–339. DOI: 10.1007/978-3-030-58604-1_20.
[2] Tang H, Li Z, Peng Z, Tang J. BlockMix: Meta regularization and self-calibrated inference for metric-based meta-learning. In Proc. the 28th ACM International Conference on Multimedia, October 2020, pp.610–618. DOI: 10.1145/3394171.3413884.
[3] Zhang D, Zhang H, Tang J, Hua X, Sun Q. Causal intervention for weakly-supervised semantic segmentation. In Proc. Annual Conference on Neural Information Processing Systems, December 2020, pp.655–666. DOI: 10.5555/3495724.3495780.
[4] Li Z, Sun Y, Zhang L, Tang J. CTNet: Context-based tandem tetwork for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9904–9917. DOI: 10.1109/TPAMI.2021.3132068.
[5] Kang L, Lin C, Fu Y. Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, 2012, 21(4): 1742–1755. DOI: 10.1109/TIP.2011.2179057.
[6] Luo Y, Xu Y, Ji H. Removing rain from a single image via discriminative sparse coding. In Proc. the 15th International Conference on Computer Vision, April 2015, pp.3397–3405. DOI: 10.1109/ICCV.2015.388.
[7] Zhu L, Fu C, Lischinski D, Heng P. Joint bi-layer optimization for single-image rain streak removal. In Proc. the 16th International Conference on Computer Vision, October 2017, pp.2545–2553. DOI: 10.1109/ICCV.2017.276.
[8] Wang Y, Liu S, Chen C, Xie D, Zeng B. Rain removal by image quasisparsity priors. arXiv: 1812.08348, 2018. https://arxiv.org/abs/1812.08348, Dec. 2018.
[9] Deng L, Huang T, Zhao X, Jiang T. A directional global sparse model for single image rain removal. Applied Mathematical Modelling, 2018, 59(7): 662–679. DOI: 10.1016/j.apm.2018.03.001.
[10] Du S, Liu Y, Ye M, Xu Z, Li J, Liu J. Single image deraining via decorrelating the rain streaks and background scene in gradient domain. Pattern Recognition, 2018, (79): 303–317. DOI: 10.1016/j.patcog.2018.02.016.
[11] Li Y, Tan T, Guo X, Lu J, Brown M. Rain streak removal using layer priors. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.2736–2744. DOI: 10.1109/CVPR.2016.299.
[12] Fu X, Huang J, Ding X, Liao Y, Paisley J. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. DOI: 10.1109/TIP.2017.2691802.
[13] Pan J, Liu S, Sun D, Zhang J, Liu Y, Ren J, Li Z, Tang J, Lu H, Tai Y, Yang M. Learning dual convolutional neural networks for low-level vision. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.3070–3079. DOI: 10.1109/CVPR.2018.00324.
[14] Wang Y, Zhao X, Jiang T, Deng L, Chang Y, Huang T. Rain streak removal for single image via kernel guided CNN. arXiv: 1808.08545, 2018. https://arxiv.org/abs/1808.08545, Aug. 2018.
[15] Ye H, Li X, Liu H, Shi W, Liu M, Sun Q. Self-rening deep symmetry enhanced network for rain removal. In Proc. the 25th IEEE International Conference on Image Processing, September 2018, pp.2786–2790. DOI: 10.1109/ICIP.2019.8803265.
[16] Li G, He X, Zhang W, Chang H, Dong L, Lin L. Non-locally enhanced encoder-decoder network for single image de-raining. In Proc. the 26th ACM International Conference on Multimedia, October 2018, 1056–1064. DOI: 10.1145/3240508.3240636.
[17] Li S, Ren W, Zhang J, Yu J, Guo X. Single image rain removal via a deep decomposition-composition network. Computer Vision and Image Understanding, 2019, 186: 48–57. DOI: 10.1016/j.cviu.2019.05.003.
[18] Yang W, Tan T, Feng J, Guo Z, Yan S, Liu J. Joint rain detection and removal from a single image with contextualized deep networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(6): 1377–1393. DOI: 10.1109/TPAMI.2019.2895793.
[19] Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J. Removing rain from single images via a deep detail network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.1715–1723. DOI: 10.1109/CVPR.2017.186.
[20] Fu X, Liang B, Huang Y, Ding X, Paisley J. Lightweight pyramid networks for image deraining. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(6): 1794–1807. DOI: 10.1109/TNNLS.2019.2926481.
[21] Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G. Gated context aggregation network for image dehazing and deraining. In Proc. the 2019 IEEE Winter Conference on Applications of Computer Vision, January 2019, pp.1375–1383. DOI: 10.1109/WACV.2019.00151.
[22] Fu X, Qi Q, Huang Y, Ding X, Wu F, Paisley J. A deep tree-structured fusion model for single image deraining. arXiv: 1811.08632, 2018. https://arxiv.org/abs/1811.08632, Nov. 2018.
[23] Ren D, Zuo W, Hu Q, Zhu P, Meng D. Progressive image deraining networks: A better and simpler baseline. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.3937–3946. DOI: 10.1109/CVPR.2019.00406.
[24] Li X, Wu J, Lin Z, Liu H, Zha H. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proc. the 32nd European Conference on Computer Vision, September 2018, pp.262–277. DOI: 10.1007/978-3-030-01234-2_16.
[25] Zhang H, Patel V. Density-aware single image de-raining using a multi-stream dense network. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.695–704. DOI: 10.1109/CVPR.2018.00079.
[26] Zhang H, Sindagi V, Patel V. Image de-raining using a conditional generative adversarial network. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3943–3956. DOI: 10.1109/TCSVT.2019.2920407.
[27] Qian R, Tan T, Yang W, Su J, Liu J. Attentive generative adversarial network for raindrop removal from a single image. In Proc. the 31st Conference on Computer Vision and Pattern Recognition, September 2018, pp.2482–2491. DOI: 10.1109/CVPR.2018.00263.
[28] Pu J, Chen X, Zhang L, Zhou Q, Zhao Y. Removing rain based on a cycle generative adversarial network. In Proc. the 13th IEEE Conference on Industrial Electronics and Applications, May 31–June 2, 2018, pp.621–626. DOI: 10.1109/ICIEA.2018.8397790.
[29] Pan J, Liu Y, Dong J, Zhang J, Ren J, Tang J, Tai Y, Yang M. Physics-based generative adversarial models for image restoration and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2449–2462. DOI: 10.1109/TPAMI.2020.2969348.
[30] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770–778. DOI: 10.1109/CVPR.2016.90.
[31] Wang D, Tang H, Pan J, Tang J. Learning a tree-structured channel-wise refinement network for efficient image deraining. In Proc. IEEE International Conference on Multimedia and Expo, July 2021. DOI: 10.1109/ICME51207.2021.9428187.
[32] Fan Z, Wu H, Fu X, Huang Y, Ding X. Residual guide network for single image deraining. In Proc. the 26th ACM International Conference on Multimedia, October 2018, pp.1751–1759. DOI: 10.1145/3240508.3240694.
[33] Wei W, Meng D, Zhao Q, Xu Z, Wu Y. Semi-supervised transfer learning for image rain removal. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.3877–3886. DOI: 10.1109/CVPR.2019.00400.
[34] Lin H, Li Y, Fu X, Ding X, Huang Y, Paisley J. Rain O'er Me: Synthesizing real rain to derain with data distillation. IEEE Transactions on Image Processing, 29(6): 7668–7680. DOI: 10.1109/TIP.2020.3005517.
[35] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.7132–7141. DOI: 10.1109/CVPR.2018.00745.
[36] Woo S, Park J, Lee J, Kweon I. CBAM: Convolutional block attention module. In Proc. the 32nd European Conference on Computer Vision, September 2018, pp.3–19. DOI: 10.1007/978-3-030-01234-21.
[37] Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, JohannesTotz, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.105–114. DOI: 10.1109/CVPR.2017.19.
[38] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A, Li F. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252. DOI: 10.1007/s11263-015-0816-y.
[39] Ledig C, Theis L, Caballero J, Cunningham A, Acosta A, Tejani A, Totz J, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.105–114. DOI: 10.1109/CVPR.2017.19.
[40] Wang T, Yang X, Xu K, Chen S, Zhang Q, Lau R. Spatial attentive single-image deraining with a high quality real rain dataset. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.12270–12279. DOI: 10.1109/CVPR.2019.01255.
[41] Quan H, Mohammed G. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 2008, 44(13): 800–801. DOI: 10.1109/CVPR.2017.19.
[42] Wang Z, Simoncelli E, Bovik A. Multiscale structural similarity for image quality assessment. In Proc. the 37th Asilomar Conference on Signals, Systems & Computers, Nov. 2003, pp.1398–1402. DOI: 10.1109/ACSSC.2003.1292216.
[43] Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv: 1412.6980, 2014. https://arxiv.org/abs/1412.6980, Jan. 2017.
[44] He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. DOI: 10.1109/TPAMI.2010.168.
[45] Li R, Cheong L, Tan T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2020, pp.1633–1642. DOI: 10.1109/CVPR.2019.00173.
[46] Liu J, Yang W, Yang S, Guo Z. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.3233–3242. DOI: 10.1109/CVPR.2018.00341.
[47] Tang H, Yuan C, Li Z, Tang J. Learning attention-guided pyramidal features for few-shot fine-grained recognition. Pattern Recognition, 2022. DOI: 10.1016/j.patcog.2022.108792. (preprint)
[48] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. DOI: 10.1109/TPAMI.2016.2577031.
[49] Lin T, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick C. Microsoft COCO: Common objects in context. In Proc. the 13th European Conference on Computer Vision, September 2014, pp.740–755. DOI: 10.1007/978-3-319-10602-148.
[50] Everingham M, Gool L, Williams C, Winn J, Zisserman A. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303–338. DOI: 10.1007/s11263-009-0275-4.
[1] Huan-Jing Yue, Sheng Shen, Jing-Yu Yang, Hao-Feng Hu, Yan-Fang Chen. Reference Image Guided Super-Resolution via Progressive Channel Attention Networks [J]. Journal of Computer Science and Technology, 2020, 35(3): 551-563.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Zhou Di;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] Li Wei;. A Structural Operational Semantics for an Edison Like Language(2)[J]. , 1986, 1(2): 42 -53 .
[3] Chen Shihua;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[4] Li Wanxue;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[5] Feng Yulin;. Recursive Implementation of VLSI Circuits[J]. , 1986, 1(2): 72 -82 .
[6] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[7] Wang Xuan; Lü Zhimin; Tang Yuhai; Xiang Yang;. A High Resolution Chinese Character Generator[J]. , 1986, 1(2): 1 -14 .
[8] C.Y.Chung; H.R.Hwa;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[9] Sun Zhongxiu; Shang Lujun;. DMODULA:A Distributed Programming Language[J]. , 1986, 1(2): 25 -31 .
[10] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved