Please wait a minute...
 
Remote Sensing for Natural Resources    2023, Vol. 35 Issue (4) : 9-16     DOI: 10.6046/zrzyyg.2022317
|
Deep learning-based cloud detection method for multi-source satellite remote sensing images
DENG Dingzhu()
Inner Mongolia Autonomous Region Surveying, Mapping and Geographic Information Center, Hohhot 010051, China
Download: PDF(4339 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Cloud detection, as a crucial step in preprocessing optical satellite images, plays a significant role in the subsequent application analysis. The increasingly enriched optical satellite remote sensing images pose a challenge in achieving quick cloud detection of numerous multi-source satellite remote sensing images. Given that conventional cloud detection exhibits low accuracy and limited universality, this study proposed a multi-scale feature fusion neural network model, i.e., the multi-source remote sensing cloud detection network (MCDNet). The MCDNet comprises a U-shaped architecture and a lightweight backbone network, and its decoder integrates multi-scale feature fusion and a channel attention mechanism to enhance model performance. The MCDNet model was trained using tens of thousands of globally distributed multi-source satellite images, covering commonly used satellite data like Google and Landsat data and domestic satellite data like GF-1, GF-2, and GF-5 data. Several classic semantic segmentation models were used for comparison with the MCDNet model in the experiment. The experimental results indicate that the MCDNet model exhibited superior performance in cloud detection, achieving detection accuracy of over 90% for all types of satellite data. Additionally, the MCDNet model was tested on the Sentinel data that were not used in training, yielding satisfactory cloud detection effects. This demonstrates the MCDNet model’s robustness and potential for use as a general model for cloud detection of medium- to high-resolution satellite images.
Keywords cloud detection      deep learning      multi-source remote sensing      domestic satellite      convolutional neural network      attention mechanism     
ZTFLH:  TP79  
  TP751.1  
Issue Date: 21 December 2023
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Dingzhu DENG
Cite this article:   
Dingzhu DENG. Deep learning-based cloud detection method for multi-source satellite remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(4): 9-16.
URL:  
https://www.gtzyyg.com/EN/10.6046/zrzyyg.2022317     OR     https://www.gtzyyg.com/EN/Y2023/V35/I4/9
Fig.1  Architecture diagram of MCDNet
序号 卫星数据类型 空间分辨率/m 影像尺寸/像素 影像数量/个 切片数量/个 参考文献
1 Google 0.5~1.5 1 280×720 150 600 [25]
2 Landsat5/7/8 30 384×384 8 400 8 400 [40]
3 GF-1 16 1 200×1 300 4 168 5 000 [41]
4 GF-2 4 7 300×6 908 34 1 000 [42]
5 GF-5 30 2 008×2 083 480 5 000 [37]
Tab.1  Dataset for cloud detection from multi-source remote sensing images
Fig.2  Training curve of the MCDNet
模型 P R F1 OA IoU
SegNet 0.89 0.83 0.86 0.93 0.75
PSPNet 0.87 0.86 0.86 0.93 0.76
HRNetV2 0.94 0.85 0.89 0.94 0.80
UNet 0.85 0.94 0.89 0.95 0.80
BiSeNet 0.84 0.96 0.89 0.95 0.81
DeeplabV3+ 0.91 0.90 0.91 0.95 0.83
MFGNet 0.91 0.91 0.91 0.96 0.84
MCDNet 0.93 0.95 0.94 0.97 0.89
Tab.2  Evaluation for cloud detection on multi-source remote sensing images
模型 P R F1 OA IoU
MCDNet-withoutAT 0.92 0.92 0.92 0.96 0.85
MCDNet-Xcep 0.91 0.96 0.93 0.97 0.87
MCDNet-withoutDC 0.92 0.95 0.93 0.97 0.88
MCDNet 0.93 0.95 0.94 0.97 0.89
Tab.3  Evaluation for ablation experiments of MCDNet
卫星 0%云覆盖 20%云覆盖
真彩色影像 云检测结果 真彩色影像 云检测结果
Landsat
GF-5
GF-2
GF-1
Google
卫星 45%云覆盖 80%云覆盖
真彩色影像 云检测结果 真彩色影像 云检测结果
Landsat
GF-5
GF-2
GF-1
Google
Tab.4  Multi-source remote sensing true color images and MCDNet cloud detection results under different cloud coverage conditions
Fig.3  Cloud detection results of MCDNet on Sentinel-2 images
[1] King M D, Platnick S, Menzel W P, et al. Spatial and temporal distribution of clouds observed by MODIS onboard the Terra and Aqua satellites[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(7):3826-3852.
doi: 10.1109/TGRS.2012.2227333 url: http://ieeexplore.ieee.org/document/6422379/
[2] Yu J, Yan B. Efficient solution of large-scale domestic hyperspectral data processing and geological application[C]// 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).IEEE, 2017:1-4.
[3] Irish R R, Barker J L, Goward S N, et al. Characterization of the Landsat7 ETM+ automated cloud-cover assessment (ACCA) algorithm[J]. Photogrammetric Engineering and Remote Sensing, 2006, 72(10):1179-1188.
doi: 10.14358/PERS.72.10.1179 url: http://openurl.ingenta.com/content/xref?genre=article&issn=0099-1112&volume=72&issue=10&spage=1179
[4] Zhu Z, Woodcock C E. Object-based cloud and cloud shadow detection in Landsat imagery[J]. Remote Sensing of Environment, 2012, 118:83-94.
doi: 10.1016/j.rse.2011.10.028 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425711003853
[5] Zhu Z, Wang S, Woodcock C E. Improvement and expansion of the Fmask algorithm:Cloud,cloud shadow,and snow detection for Landsats4-7,8,and Sentinel-2 images[J]. Remote Sensing of Environment, 2015, 159:269-277.
doi: 10.1016/j.rse.2014.12.014 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425714005069
[6] Rossow W B, Garder L C. Cloud detection using satellite measurements of infrared and visible radiances for ISCCP[J]. Journal of Climate, 1993, 6(12):2341-2369.
doi: 10.1175/1520-0442(1993)006<2341:CDUSMO>2.0.CO;2 url: http://journals.ametsoc.org/doi/10.1175/1520-0442(1993)006<2341:CDUSMO>2.0.CO;2
[7] Gesell G. An algorithm for snow and ice detection using AVHRR data:An extension to the APOLLO software package[J]. International Journal of Remote Sensing, 1989, 10(4/5):897-905.
doi: 10.1080/01431168908903929 url: https://www.tandfonline.com/doi/full/10.1080/01431168908903929
[8] Stowe L L, McClain E P, Carey R, et al. Global distribution of cloud cover derived from NOAA/AVHRR operational satellite data[J]. Advances in Space Research, 1991, 11(3):51-54.
[9] Qiu S, He B, Zhu Z, et al. Improving Fmask cloud and cloud shadow detection in mountainous area for Landsats4-8 images[J]. Remote Sensing of Environment, 2017, 199:107-119.
doi: 10.1016/j.rse.2017.07.002 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425717303073
[10] Qiu S, Zhu Z, He B. Fmask 4.0:Improved cloud and cloud shadow detection in Landsats4-8 and Sentinel-2 imagery[J]. Remote Sensing of Environment, 2019, 231:111205.
doi: 10.1016/j.rse.2019.05.024 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425719302172
[11] Zhu X, Helmer E H. An automatic method for screening clouds and cloud shadows in optical satellite time series in cloudy regions[J]. Remote Sensing of Environment, 2018, 214:135-153.
doi: 10.1016/j.rse.2018.05.024 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425718302530
[12] Frantz D, Haß E, Uhl A, et al. Improvement of the Fmask algorithm for Sentinel-2 images:Separating clouds from bright surfaces based on parallax effects[J]. Remote Sensing of Environment, 2018, 215:471-481.
doi: 10.1016/j.rse.2018.04.046 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425718302037
[13] Bian J, Li A, Liu Q, et al. Cloud and snow discrimination for CCD images of HJ-1A/B constellation based on spectral signature and spatio-temporal context[J]. Remote Sensing, 2016, 8(1):31.
doi: 10.3390/rs8010031 url: http://www.mdpi.com/2072-4292/8/1/31
[14] 葛曙乐, 董胜越, 孙根云, 等. 一种适用于高分五号全谱段光谱成像仪影像的云检测算法[J]. 上海航天, 2019, 36(s2):204-209.
[14] Ge S L, Dong S Y, Sun G Y, et al. Cloud detection algorithm for images of visual and infrared multispectral imager[J]. Aerospace Shanghai, 2019, 36(s2):204-209.
[15] Zhan Y, Wang J, Shi J, et al. Distinguishing cloud and snow in satellite images via deep convolutional network[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10):1785-1789.
doi: 10.1109/LGRS.2017.2735801 url: http://ieeexplore.ieee.org/document/8013916/
[16] Wang L, Chen Y, Tang L, et al. Object-based convolutional neural networks for cloud and snow detection in high-resolution multispectral imagers[J]. Water, 2018, 10(11):1666.
doi: 10.3390/w10111666 url: http://www.mdpi.com/2073-4441/10/11/1666
[17] Oishi Y, Ishida H, Nakamura R. A new Landsat8 cloud discrimination algorithm using thresholding tests[J]. International Journal of Remote Sensing, 2018, 39(23):9113-9133.
doi: 10.1080/01431161.2018.1506183 url: https://www.tandfonline.com/doi/full/10.1080/01431161.2018.1506183
[18] Shao Z, Pan Y, Diao C, et al. Cloud detection in remote sensing images based on multiscale features-convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6):4062-4076.
doi: 10.1109/TGRS.36 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=36
[19] Hong Y, Hsu K L, Sorooshian S, et al. Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system[J]. Journal of Applied Meteorology, 2004, 43(12):1834-1853.
doi: 10.1175/JAM2173.1 url: https://journals.ametsoc.org/doi/10.1175/JAM2173.1
[20] Hall D K, Riggs G A, Salomonson V V. Development of methods for mapping global snow cover using moderate resolution imaging spectroradiometer data[J]. Remote Sensing of Environment, 1995, 54(2):127-140.
doi: 10.1016/0034-4257(95)00137-P url: https://linkinghub.elsevier.com/retrieve/pii/003442579500137P
[21] Ghasemian N, Akhoondzadeh M. Introducing two random forest based methods for cloud detection in remote sensing images[J]. Advances in Space Research, 2018, 62(2):288-303.
doi: 10.1016/j.asr.2018.04.030 url: https://linkinghub.elsevier.com/retrieve/pii/S0273117718303624
[22] Egli S, Thies B, Bendix J. A hybrid approach for fog retrieval based on a combination of satellite and ground truth data[J]. Remote Sensing, 2018, 10(4):628.
doi: 10.3390/rs10040628 url: https://www.mdpi.com/2072-4292/10/4/628
[23] Lee Y, Wahba G, Ackerman S A. Cloud classification of satellite radiance data by multicategory support vector machines[J]. Journal of Atmospheric and Oceanic Technology, 2004, 21(2):159-169.
doi: 10.1175/1520-0426(2004)021<0159:CCOSRD>2.0.CO;2 url: http://journals.ametsoc.org/doi/10.1175/1520-0426(2004)021<0159:CCOSRD>2.0.CO;2
[24] Ishida H, Oishi Y, Morita K, et al. Development of a support vector machine based cloud detection method for MODIS with the adjustability to various conditions[J]. Remote Sensing of Environment, 2018, 205:390-407.
doi: 10.1016/j.rse.2017.11.003 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425717305138
[25] Li Z, Shen H, Cheng Q, et al. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 150:197-212.
doi: 10.1016/j.isprsjprs.2019.02.017 url: https://linkinghub.elsevier.com/retrieve/pii/S0924271619300565
[26] Ma L, Liu Y, Zhang X, et al. Deep learning in remote sensing applications:A meta-analysis and review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 152:166-177.
doi: 10.1016/j.isprsjprs.2019.04.015 url: https://linkinghub.elsevier.com/retrieve/pii/S0924271619301108
[27] Yu J, Zhang L, Li Q, et al. 3D autoencoder algorithm for lithological mapping using ZY-1 02D hyperspectral imagery:A case study of Liuyuan region[J]. Journal of Applied Remote Sensing, 2021, 15(4):042610.
[28] Tsagkatakis G, Aidini A, Fotiadou K, et al. Survey of deep-learning approaches for remote sensing observation enhancement[J]. Sensors, 2019, 19(18):3929.
doi: 10.3390/s19183929 url: https://www.mdpi.com/1424-8220/19/18/3929
[29] Yang J, Guo J, Yue H, et al. CDnet:CNN-based cloud detection for remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8):6195-6211.
doi: 10.1109/TGRS.36 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=36
[30] Jeppesen J H, Jacobsen R H, Inceoglu F, et al. A cloud detection algorithm for satellite imagery based on deep learning[J]. Remote Sensing of Environment, 2019, 229:247-259.
doi: 10.1016/j.rse.2019.03.039
[31] Drönner J, Korfhage N, Egli S, et al. Fast cloud segmentation using convolutional neural networks[J]. Remote Sensing, 2018, 10(11):1782.
doi: 10.3390/rs10111782 url: http://www.mdpi.com/2072-4292/10/11/1782
[32] 刘广进, 王光辉, 毕卫华, 等. 基于DenseNet与注意力机制的遥感影像云检测算法[J]. 自然资源遥感, 2022, 34(2):88-96.doi:10.6046/zrzyyg.2021128.
[32] Liu G J, Wang G H, Bi W H, et al. Cloud detection algorithm of remote sensing image based on DenseNet and attention mechanism[J]. Remote Sensing for Natural Resources, 2022, 34(2):88-96.doi:10.6046/zrzyyg.2021128.
[33] Chai D, Newsam S, Zhang H K, et al. Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks[J]. Remote Sensing of Environment, 2019, 225:307-316.
doi: 10.1016/j.rse.2019.03.007 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425719300987
[34] Mohajerani S, Saeedi P. Cloud-net+:A cloud segmentation CNN for Landsat8 remote sensing imagery optimized with filtered jaccard loss function[J/OL]. arXiv, 2020(2020-01-23)[2021-04-23]. https://arxiv.org/abs/2001.08768v1.
url: https://arxiv.org/abs/2001.08768v1
[35] Guo Z S, Li C H, Wang Z M, et al. A cloud boundary detection scheme combined with ASLIC and CNN using ZY-3,GF-1/2 satellite imagery[J]. International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences, 2018, 42(3):455-458.
[36] Zi Y, Xie F, Jiang Z. A cloud detection method for Landsat8 images based on PCANet[J]. Remote Sensing, 2018, 10(6):877.
doi: 10.3390/rs10060877 url: http://www.mdpi.com/2072-4292/10/6/877
[37] Yu J C, Li Y, Zheng X, et al. An effective cloud detection method for Gaofen-5 images via deep learning[J]. Remote Sensing, 2020, 12(13):2106.
doi: 10.3390/rs12132106 url: https://www.mdpi.com/2072-4292/12/13/2106
[38] 仇一帆, 柴登峰. 无人工标注数据的Landsat影像云检测深度学习方法[J]. 国土资源遥感, 2021, 33(1): 102-107.doi:10.6046/gtzyyg.2020090.
[38] Qiu Y F, Chai D F. A deep learning method for Landsat image cloud detection without manually labeled data[J]. Remote Sensing for Land and Resources, 2021, 33(1): 102-107.doi:10.6046/gtzyyg.2020090.
[39] Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham:Springer, 2015:234-241.
[40] Mohajerani S, Saeedi P. Cloud-Net:An end-to-end cloud detection algorithm for Landsat8 imagery[C]// 2019 IEEE International Geoscience and Remote Sensing Symposium.IEEE, 2019: 1029-1032.
[41] Wu X, Shi Z, Zou Z. A geographic information-driven method and a new large scale dataset for remote sensing cloud/snow detection[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 174:87-104.
doi: 10.1016/j.isprsjprs.2021.01.023 url: https://linkinghub.elsevier.com/retrieve/pii/S0924271621000290
[42] He Q, Sun X, Yan Z, et al. DABNet:Deformable contextual and boundary-weighted network for cloud detection in remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:5601216.
[43] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39:2481-2495.
doi: 10.1109/TPAMI.34 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=34
[44] Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017:6230-6239.
[45] Yu C, Wang J, Peng C, et al. Bisenet:Bilateral segmentation network for real-time semantic segmentation[C]// 15th European Conference on Computer Vision (ECCV), 2018:334-349.
[46] Sun K, Zhao Y, Jiang B, et al. High-resolution representations for labeling pixels and regions[J/OL]. arXiv, 2019(2019-04-09)[2021-04-23]. https://arxiv.org/abs/1904.04514.pdf.
url: https://arxiv.org/abs/1904.04514.pdf
[47] Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// 15th European Conference on Computer Vision (ECCV), 2018:833-851.
[1] LI Xintong, SHI Lan, CHEN Duoyan. A deep learning-based study on downscaling of GPM products in Fujian-Zhejiang-Jiangxi area[J]. Remote Sensing for Natural Resources, 2023, 35(4): 105-113.
[2] CHEN Di, PENG Qiuzhi, HUANG Peiyi, LIU Yaxuan. Detecting land for photovoltaic development based on the attention mechanism and improved YOLOv5[J]. Remote Sensing for Natural Resources, 2023, 35(4): 90-95.
[3] LIU Hanwei, CHEN Fulong, LIAO Yaao. Remote sensing dynamic monitoring and driving factor analysis for the Beijing section of Ming Great Wall[J]. Remote Sensing for Natural Resources, 2023, 35(4): 255-263.
[4] LIU Li, DONG Xianmin, LIU Juan. A performance evaluation method for semantic segmentation models of remote sensing images considering surface features[J]. Remote Sensing for Natural Resources, 2023, 35(3): 80-87.
[5] NIU Xianghua, HUANG Wei, HUANG Rui, JIANG Sili. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.
[6] WU Weichao, YE Fawang. Cloud detection of Sentinel-2 images for multiple backgrounds[J]. Remote Sensing for Natural Resources, 2023, 35(3): 124-133.
[7] ZHANG Xian, LI Wei, CHEN Li, YANG Zhaoying, DOU Baocheng, LI Yu, CHEN Haomin. Research progress and prospect of remote sensing-based feature extraction of opencast mining areas[J]. Remote Sensing for Natural Resources, 2023, 35(2): 25-33.
[8] DIAO Mingguang, LIU Yong, GUO Ningbo, LI Wenji, JIANG Jikang, WANG Yunxiao. Mask R-CNN-based intelligent identification of sparse woods from remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 97-104.
[9] ZHENG Zongsheng, LIU Haixia, WANG Zhenhua, LU Peng, SHEN Xukun, TANG Pengfei. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 105-111.
[10] QIU Lei, ZHANG Xuezhi, HAO Dawei. VideoSAR moving target detection and tracking algorithm based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(2): 157-166.
[11] ZHANG Ke, ZHANG Gengsheng, WANG Ning, WEN Jing, LI Yu, YANG Jun. A forecasting method for water table depths in areas with power transmission lines based on remote sensing and deep learning models[J]. Remote Sensing for Natural Resources, 2023, 35(1): 213-221.
[12] SUN Sheng, MENG Zhimin, HU Zhongwen, YU Xu. Application of multi-scale and lightweight CNN in SAR image-based surface feature classification[J]. Remote Sensing for Natural Resources, 2023, 35(1): 27-34.
[13] JIN Yuanhang, XU Maolin, ZHENG Jiayuan. A dead tree detection algorithm based on improved YOLOv4-tiny for UAV images[J]. Remote Sensing for Natural Resources, 2023, 35(1): 90-98.
[14] HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[15] ZHAO Linghu, YUAN Xiping, GAN Shu, HU Lin, QIU Mingyu. An information extraction model of roads from high-resolution remote sensing images based on improved Deeplabv3+[J]. Remote Sensing for Natural Resources, 2023, 35(1): 107-114.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech