计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (3): 622-633.doi: 10.1007/s11390-019-1931-y

所属专题: Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • 上一篇    下一篇

基于超像素分类和语义分割的云检测算法

Han Liu1,2, Hang Du1,2, Dan Zeng1,2,*, Qi Tian3, Fellow, IEEE   

  1. 1 Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China;
    2 Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China;
    3 Department of Computer Science, The University of Texas at San Antonio, San Antonio, U.S.A.
  • 收稿日期:2018-10-19 修回日期:2019-03-27 出版日期:2019-05-05 发布日期:2019-05-06
  • 通讯作者: Dan Zeng E-mail:dzeng@shu.edu.cn
  • 作者简介:Han Liu received his B.E. degree in communication and information engineering from Shanghai University, Shanghai, in 2017. He is currently pursuing his M.E. degree at Key Laboratory of Specialty Fiber Optics and Optical Access Networks and Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai. His research interests focus on computer vision and pattern recognition.
  • 基金资助:
    This work was supported in part by ARO under Grant No. W911NF-15-1-0290 and Faculty Research Gift Awards by NEC Laboratories of America and Blippar to Dr. Qi Tian. It was also supported in part by the National Natural Science Foundation of China under Grant Nos. 61429201 and 61572307.

Cloud Detection Using Super Pixel Classification and Semantic Segmentation

Han Liu1,2, Hang Du1,2, Dan Zeng1,2,*, Qi Tian3, Fellow, IEEE   

  1. 1 Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China;
    2 Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China;
    3 Department of Computer Science, The University of Texas at San Antonio, San Antonio, U.S.A.
  • Received:2018-10-19 Revised:2019-03-27 Online:2019-05-05 Published:2019-05-06
  • Contact: Dan Zeng E-mail:dzeng@shu.edu.cn
  • About author:Han Liu received his B.E. degree in communication and information engineering from Shanghai University, Shanghai, in 2017. He is currently pursuing his M.E. degree at Key Laboratory of Specialty Fiber Optics and Optical Access Networks and Shanghai Institute of Advanced Communication and Data Science, Shanghai University, Shanghai. His research interests focus on computer vision and pattern recognition.
  • Supported by:
    This work was supported in part by ARO under Grant No. W911NF-15-1-0290 and Faculty Research Gift Awards by NEC Laboratories of America and Blippar to Dr. Qi Tian. It was also supported in part by the National Natural Science Foundation of China under Grant Nos. 61429201 and 61572307.

云检测在遥感图像处理中起着非常重要的作用。本文介绍了一种基于超像素图像分类和语义分割的云检测方法。首先,利用SLIC和SEEDS将遥感图像分割成超像素。超像素构成超像素级遥感图像数据集。虽然云检测本质上是一个二分类任务,但我们的数据集中的图像被分为四类以增强泛化能力:厚云,卷云,建筑和其他地物。其次,超像素级数据集用于训练基于卷积神经网络和深度森林的云检测模型。考虑到超像素图像比普通图像包含更少的语义信息,本文提出分层融合卷积神经网络,其充分利用颜色和纹理信息等低层特征,更适用于超级像素级别的分类任务。此外,本文提出距离度量来细化类别模糊的超像素。第三,本文介绍了一种基于语义分割的端到端的云检测模型。该模型对输入图像大小没有限制,并且消耗时间更少。实验结果表明,与其他云检测方法相比,我们提出的方法可以取得更好的性能。

关键词: 云检测, 卷积神经网络, 深度森林, 语义分割

Abstract: Cloud detection plays a very significant role in remote sensing image processing. This paper introduces a cloud detection method based on super pixel level classification and semantic segmentation. Firstly, remote sensing images are segmented into super pixels. Segmented super pixels compose a super pixel level remote sensing image database. Though cloud detection is essentially a binary classification task, our database is labeled into four categories to improve the generalization ability:thick cloud, cirrus cloud, building, and other culture. Secondly, the super pixel level database is used to train our cloud detection models based on convolution neural network (CNN) and deep forest. Hierarchical fusion CNN is proposed considering super pixel level images contain less semantic information than normal images. Taking full advantage of low-level features like color and texture information, it is more applicable for super pixel level classification. Besides, a distance metric is proposed to refine ambiguous super pixels. Thirdly, an end-to-end cloud detection model based on semantic segmentation is introduced. This model has no restrictions on the input size, and takes less time. Experimental results show that compared with other cloud detection methods, our proposed method achieves better performance.

Key words: cloud detection, convolution neural network, deep forest, semantic segmentation

[1] Xie F, Shi M, Shi Z, Yin J, Zhao D. Multilevel cloud detection in remote sensing images based on deep learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3631-3640.
[2] Shi M, Xie F, Zi Y, Yin J. Cloud detection of remote sensing images by deep learning. In Proc. the 2016 IEEE International Geoscience and Remote Sensing Symposium, July 2016, pp.701-704.
[3] Liu H, Zeng D, Tian Q. Super pixel cloud detection using hierarchical fusion CNN. In Proc. the 4th IEEE International Conference on Multimedia Big Data, September 2018, Article No. 32.
[4] Zhou Z H, Feng J. Deep forest:Towards an alternative to deep neural networks. arXiv:1702.08835, 2017. https://arxiv.org/abs/1702.08835, May 2018.
[5] Jedlovec G J, Haines S L, LaFontaine F J. Spatial and temporal varying thresholds for cloud detection in GOES imagery. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(6):1705-1717.
[6] Zhang Q, Xiao C. Cloud detection of RGB color aerial photographs by progressive refinement scheme. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(11):7264-7275.
[7] Zhong B, Chen W, Wu S, Hu L, Luo X, Liu Q. A cloud detection method based on relationship between objects of cloud and cloud-shadow for Chinese moderate to high resolution satellite imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(11):4898-4908.
[8] Li Z, Shen H, Li H, Xia G, Gamba P, Zhang L. Multi-feature combined cloud and cloud shadow detection in GaoFen-1 wide field of view imagery. Remote Sensing of Environment, 2017, 191:342-358.
[9] Visa A, Valkealahti K, Simula O. Cloud detection based on texture segmentation by neural network methods. In Proc. the 1991 IEEE International Joint Conference on Neural Networks, Vol. 2, November 1991, pp.1001-1006.
[10] Zhan Y, Wang J, Shi J, Cheng G, Yao L, Sun W. Distinguishing cloud and snow in satellite images via deep convolutional network. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10):1785-1789.
[11] Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11):2274-2282.
[12] Hu X, Wang Y, Shan J. Automatic recognition of cloud images by using visual saliency features. IEEE Geosci. Remote Sensing Lett., 2015, 12(8):1760-1764.
[13] Tan K, Zhang Y, Tong X. Cloud extraction from Chinese high resolution satellite imagery by probabilistic latent semantic analysis and object-based machine learning. Remote Sensing, 2016, 8(11):Article No. 963.
[14] Yuan Y, Hu X. Bag-of-words and object-based classification for cloud extraction from satellite imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(8):4197-4205.
[15] Li Z, Shen H, Wei Y, Cheng Q, Yuan Q. Cloud detection by fusing multi-scale convolutional features. In Proc. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Science, May 2018, pp.149-152.
[16] Ma C, Luo G, Wang K. Concatenated and connected random forests with multiscale patch driven active contour model for automated brain tumor segmentation of MR images. IEEE Transactions on Medical Imaging, 2018, 37(8):1943-1954.
[17] Zhang C, Yan J, Li C, Bie R. Contour detection via stacking random forest learning. Neurocomputing, 2018, 275:2702- 2715.
[18] Shen Y, Lu H, Jia J. Classification of motor imagery EEG signals with deep learning models. In Proc. the 7th International Conference on Intelligent Science and Big Data Engineering, September 2017, pp.181-190.
[19] Dollár P, Zitnick C L. Fast edge detection using structured forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(8):1558-1570.
[20] van den Bergh M, Boix X, Roig G, van Gool L. SEEDS:Superpixels extracted via energy-driven sampling. International Journal of Computer Vision, 2015, 111(3):298-314.
[21] Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009. http://www.cs.toronto.edu/~kriz/learning-features- 2009-TR.pdf, March 2019.
[22] Lin T Y, Dollár P, Girshick R B, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.936-944.
[23] Chen L C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. arXiv:1802.02611, 2018. https://arxiv.org/abs/1802.02611, March 2019.
[24] Chen L C, Papandreou G, Schroff F, Adam H. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017. https://arxiv.org/abs/1706.05587, March 2019.
[25] Chollet F. Xception:Deep learning with depthwise separable convolutions. arXiv:1610.02357, 2016. https://arxiv.org/abs/1610.02357, March 2019.
[26] Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe:Convolutional architecture for fast feature embedding. In Proc. the 22nd ACM International Conference on Multimedia, November 2014, pp.675-678.
[27] Tang M, Gorelick L, Veksler O, Boykov Y. GrabCut in one cut. In Proc. the 2013 IEEE International Conference on Computer Vision, December 2013, pp.1769-1776.
[1] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[2] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器:利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[3] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗?[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[4] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[5] 梁盾, 郭元晨, 张少魁, 穆太江, 黄晓蕾. 车道检测-新结果和调查研究[J]. 计算机科学技术学报, 2020, 35(3): 493-505.
[6] Jin-Gong Jia, Yuan-Feng Zhou, Xing-Wei Hao, Feng Li, Christian Desrosiers, Cai-Ming Zhang. 双流时间卷积神经网络用于基于骨架的人体动作识别[J]. 计算机科学技术学报, 2020, 35(3): 538-550.
[7] Rui-Song Zhang, Wei-Ze Quan, Lu-Bin Fan, Li-Ming Hu, Dong-Ming Yan. 基于通道和像素相关性的计算机生成图像与自然图像鉴别[J]. 计算机科学技术学报, 2020, 35(3): 592-602.
[8] Robail Yasrab. SRNET:用于解析奇点的基于浅跳跃连接的卷积神经网络[J]. 计算机科学技术学报, 2019, 34(4): 924-938.
[9] Ri-Sheng Liu, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li. 基于灵活稀疏结构控制和自适应优化算法的模糊图像盲复原[J]. 计算机科学技术学报, 2019, 34(3): 609-621.
[10] Dong-Di Zhao, Fan Li, Kashif Sharif, Guang-Min Xia, Yu Wang. 深度卷积神经网络的空间高效量化[J]. 计算机科学技术学报, 2019, 34(2): 305-317.
[11] Tie-Ke He, Hao Lian, Ze-Min Qin, Zhen-Yu Chen, Bin Luo. 一种用于罚金判定的主题模型[J]. , 2018, 33(4): 756-767.
[12] Bei-Ji Zou, Yun-Di Guo, Qi He, Ping-Bo Ouyang, Ke Liu, Zai-Liang Chen. 基于块匹配三维滤波和卷积神经网络的图像去噪[J]. , 2018, 33(4): 838-848.
[13] Zhi-Feng Xie, Yu-Chen Guo, Shu-Han Zhang, Wen-Jun Zhang, Li-Zhuang Ma. 基于深度卷积网络的多重曝光运动估计[J]. , 2018, 33(3): 487-501.
[14] Nai-Ming Yao, Hui Chen, Qing-Pei Guo, Hong-An Wang. 使用一种基于Depth-Patch的深度神经网络的非正面人脸表情识别[J]. , 2017, 32(6): 1172-1185.
[15] Xiang Bai, Zheng Zhang, Hong-Yang Wang, Wei Shen. 指向性边缘框:基于内法向特征的一般物体检测[J]. , 2017, 32(4): 701-713.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: