Journal of Computer Science and Technology ›› 2019, Vol. 34 ›› Issue (3): 622-633.doi: 10.1007/s11390-019-1931-y

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • Previous Articles     Next Articles

Cloud Detection Using Super Pixel Classification and Semantic Segmentation

Han Liu1,2, Hang Du1,2, Dan Zeng1,2,*, Qi Tian3, Fellow, IEEE   

  1. 1 Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China;
    2 Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China;
    3 Department of Computer Science, The University of Texas at San Antonio, San Antonio, U.S.A.
  • Received:2018-10-19 Revised:2019-03-27 Online:2019-05-05 Published:2019-05-06
  • Contact: Dan Zeng E-mail:dzeng@shu.edu.cn
  • About author:Han Liu received his B.E. degree in communication and information engineering from Shanghai University, Shanghai, in 2017. He is currently pursuing his M.E. degree at Key Laboratory of Specialty Fiber Optics and Optical Access Networks and Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai. His research interests focus on computer vision and pattern recognition.
  • Supported by:
    This work was supported in part by ARO under Grant No. W911NF-15-1-0290 and Faculty Research Gift Awards by NEC Laboratories of America and Blippar to Dr. Qi Tian. It was also supported in part by the National Natural Science Foundation of China under Grant Nos. 61429201 and 61572307.

PDF

434

Supplement

1

Cloud detection plays a very significant role in remote sensing image processing. This paper introduces a cloud detection method based on super pixel level classification and semantic segmentation. Firstly, remote sensing images are segmented into super pixels. Segmented super pixels compose a super pixel level remote sensing image database. Though cloud detection is essentially a binary classification task, our database is labeled into four categories to improve the generalization ability:thick cloud, cirrus cloud, building, and other culture. Secondly, the super pixel level database is used to train our cloud detection models based on convolution neural network (CNN) and deep forest. Hierarchical fusion CNN is proposed considering super pixel level images contain less semantic information than normal images. Taking full advantage of low-level features like color and texture information, it is more applicable for super pixel level classification. Besides, a distance metric is proposed to refine ambiguous super pixels. Thirdly, an end-to-end cloud detection model based on semantic segmentation is introduced. This model has no restrictions on the input size, and takes less time. Experimental results show that compared with other cloud detection methods, our proposed method achieves better performance.

Key words: cloud detection; convolution neural network; deep forest; semantic segmentation;

[1] Xie F, Shi M, Shi Z, Yin J, Zhao D. Multilevel cloud detection in remote sensing images based on deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3631-3640.
[2] Shi M, Xie F, Zi Y, Yin J. Cloud detection of remote sensing images by deep learning. In Proc. the 2016 IEEE International Geoscience and Remote Sensing Symposium, July 2016, pp.701-704.
[3] Liu H, Zeng D, Tian Q. Super pixel cloud detection using hierarchical fusion CNN. In Proc. the 4th IEEE International Conference on Multimedia Big Data, September 2018, Article No. 32.
[4] Zhou Z H, Feng J. Deep forest:Towards an alternative to deep neural networks. arXiv:1702.08835, 2017. https://arxiv.org/abs/1702.08835, May 2018.
[5] Jedlovec G J, Haines S L, LaFontaine F J. Spatial and temporal varying thresholds for cloud detection in GOES imagery. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(6):1705-1717.
[6] Zhang Q, Xiao C. Cloud detection of RGB color aerial photographs by progressive refinement scheme. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(11):7264-7275.
[7] Zhong B, Chen W, Wu S, Hu L, Luo X, Liu Q. A cloud detection method based on relationship between objects of cloud and cloud-shadow for Chinese moderate to high resolution satellite imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(11):4898-4908.
[8] Li Z, Shen H, Li H, Xia G, Gamba P, Zhang L. Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery. Remote Sensing of Environment, 2017, 191:342-358.
[9] Visa A, Valkealahti K, Simula O. Cloud detection based on texture segmentation by neural network methods. In Proc. the 1991 IEEE International Joint Conference on Neural Networks, Vol. 2, November 1991, pp.1001-1006.
[10] Zhan Y, Wang J, Shi J, Cheng G, Yao L, Sun W. Distinguishing cloud and snow in satellite images via deep convolutional network. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10):1785-1789.
[11] Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11):2274-2282.
[12] Hu X, Wang Y, Shan J. Automatic recognition of cloud images by using visual saliency features. IEEE Geosci. Remote Sensing Lett., 2015, 12(8):1760-1764.
[13] Tan K, Zhang Y, Tong X. Cloud extraction from Chinese high resolution satellite imagery by probabilistic latent semantic analysis and object-based machine learning. Remote Sensing, 2016, 8(11):Article No. 963.
[14] Yuan Y, Hu X. Bag-of-words and object-based classification for cloud extraction from satellite imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(8):4197-4205.
[15] Li Z, Shen H, Wei Y, Cheng Q, Yuan Q. Cloud detection by fusing multi-scale convolutional features. In Proc. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Science, May 2018, pp.149-152.
[16] Ma C, Luo G, Wang K. Concatenated and connected random forests with multiscale patch driven active contour model for automated brain tumor segmentation of MR images. IEEE Transactions on Medical Imaging, 2018, 37(8):1943-1954.
[17] Zhang C, Yan J, Li C, Bie R. Contour detection via stacking random forest learning. Neurocomputing, 2018, 275:2702- 2715.
[18] Shen Y, Lu H, Jia J. Classification of motor imagery EEG signals with deep learning models. In Proc. the 7th International Conference on Intelligent Science and Big Data Engineering, September 2017, pp.181-190.
[19] Dollár P, Zitnick C L. Fast edge detection using structured forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(8):1558-1570.
[20] van den Bergh M, Boix X, Roig G, van Gool L. SEEDS:Superpixels extracted via energy-driven sampling. International Journal of Computer Vision, 2015, 111(3):298-314.
[21] Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009. http://www.cs.toronto.edu/～kriz/learning-features- 2009-TR.pdf, March 2019.
[22] Lin T Y, Dollár P, Girshick R B, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.936-944.
[23] Chen L C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv:1802.02611, 2018. https://arxiv.org/abs/1802.02611, March 2019.
[24] Chen L C, Papandreou G, Schroff F, Adam H. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017. https://arxiv.org/abs/1706.05587, March 2019.
[25] Chollet F. Xception:Deep learning with depthwise separable convolutions. arXiv:1610.02357, 2016. https://arxiv.org/abs/1610.02357, March 2019.
[26] Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe:Convolutional architecture for fast feature embedding. In Proc. the 22nd ACM International Conference on Multimedia, November 2014, pp.675-678.
[27] Tang M, Gorelick L, Veksler O, Boykov Y. GrabCut in one cut. In Proc. the 2013 IEEE International Conference on Computer Vision, December 2013, pp.1769-1776.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] Li Minghui;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved