A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion

doi:10.6046/zrzyyg.2022202

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(9361 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The thin cloud removal from remote sensing images with uneven thin cloud cover suffers from undercorrection or color distortion. This study proposed a high-fidelity end-to-end network method for thin cloud removal based on attentional feature fusion. First, this study designed an attentional feature fusion module integrating the attention mechanism and a fusion module. Through the cascade of three attentional feature fusion modules, the network focused on extracting the information on thin-cloud cover areas, reducing the impact of cloud-free areas. Furthermore, this study improved the color fidelity and detail clarity of images using the color and sharpening loss functions. The experimental results show that this method outperformed other methods in visual and quantitative evaluation indices (peak signal-to-noise ratio and structural similarity). This method yielded satisfactory effects of cloud removal in images with uneven thin cloud cover in various scenarios, producing images with actual colors, smooth brightness transition, and distinct detail contours.

Keywords thin cloud removal attention mechanism feature fusion remote sensing image deep learning

ZTFLH:

TP751

Issue Date: 19 September 2023

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Xianghua NIU
	Wei HUANG
	Rui HUANG
	Sili JIANG

Cite this article:

Xianghua NIU,Wei HUANG,Rui HUANG, et al. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2022202 OR https://www.gtzyyg.com/EN/Y2023/V35/I3/116

Fig.1 Overall structure of the proposed network

Fig.2 Structure of feature attention module

Fig.3 Structure of attention modules

Fig.4 Structure of feature fusion module

Tab.1 Cloud removal results on RICE dataset

Tab.2 Cloud removal results on thickness distribution

Tab.3 Cloud removal results on real images

Tab.4 Comparison of quality evaluation indicatorswith different methods

Fig.5 Ablation experiment of information enhancement module

Fig.6 Ablation experiment of color loss function and sharpening loss function

Fig.7 Comparison of network indoors and outdoors

[1]	He K, Sun J, Tang X. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12):2341-2353. doi: 10.1109/TPAMI.2010.168 url: http://ieeexplore.ieee.org/document/5567108/
[2]	Narasimhan S G, Nayar S K. Contrast restoration of weather degraded images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(6):713-724. doi: 10.1109/TPAMI.2003.1201821 url: http://ieeexplore.ieee.org/document/1201821/
[3]	Li J, Hu Q, Ai M. Haze and thin cloud removal via sphere model improved dark channel prior[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 16(3):472-476. doi: 10.1109/LGRS.2018.2874084 url: https://ieeexplore.ieee.org/document/8500152/
[4]	Shi S, Zhang Y, Zhou X, et al. A novel thin cloud removal method based on multiscale dark channel prior (MDCP)[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19:1-5.
[5]	Li Y, Ren J, Huang Y. An end-to-end system for unmanned aerial vehicle high-resolution remote sensing image haze removal algorithm using convolution neural network[J]. IEEE Access, 2020, 8:158787-158797. doi: 10.1109/Access.6287639 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6287639
[6]	Pan H. Cloud removal for remote sensing imagery via spatial attention generative adversarial network[J/OL]. arXiv, 2020(2020-11-14). https://arxiv.org/abs/2009.13015. url: https://arxiv.org/abs/2009.13015
[7]	Li J, Wu Z, Hu Z, et al. Thin cloud removal in optical remote sensing images based on generative adversarial networks and physical model of cloud distortion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166:373-389. doi: 10.1016/j.isprsjprs.2020.06.021 url: https://linkinghub.elsevier.com/retrieve/pii/S0924271620301787
[8]	Zi Y, Xie F, Zhang N, et al. Thin cloud removal for multispectral remote sensing images using convolutional neural networks combined with an imaging model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:3811-3823. doi: 10.1109/JSTARS.2021.3068166 url: https://ieeexplore.ieee.org/document/9384224/
[9]	Luo X, Rong W, Zhou J. A spatial-spectral adaptive thin-cloud removal method based on slow feature analysis[J]. Remote Sensing Letters, 2022, 13(8):747-755. doi: 10.1080/2150704X.2022.2079387 url: https://www.tandfonline.com/doi/full/10.1080/2150704X.2022.2079387
[10]	Woo S, Park J, Lee J Y, et al. Cbam:Convolutional block attention module[C]// Proceedings of the European Conference on Computer Vision (ECCV). 2018:3-19.
[11]	张焕, 张庆, 于纪言. 激活函数的发展综述及其性质分析[J]. 西华大学学报(自然科学版), 2021, 40(4):1-10.
[11]	Zhang H, Zhang Q, Yu J Y. Overview of the development of activation function and its nature analysis[J]. Journal of Xihua University (Natural Science Edition), 2021, 40(4):1-10.
[12]	Gupta D, Hazarika B B, Berlin M. Robust regularized extreme learning machine with asymmetric Huber loss function[J]. Neural Computing and Applications, 2020, 32(16):12971-12998. doi: 10.1007/s00521-020-04741-w
[13]	Pang B, Nijkamp E, Wu Y N. Deep learning with tensorflow:A review[J]. Journal of Educational and Behavioral Statistics, 2020, 45(2):227-248. doi: 10.3102/1076998619872761
[14]	Lin D, Xu G, Wang X, et al. A remote sensing image dataset for cloud removal[J/OL]. arXiv, 2019(2019-01-03). http://arxiv.org/abs/1901.00600v1. url: http://arxiv.org/abs/1901.00600v1
[15]	Pan X, Xie F, Jiang Z, et al. No-reference assessment on haze for remote-sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(12):1855-1859. doi: 10.1109/LGRS.2016.2614890 url: http://ieeexplore.ieee.org/document/7600445/
[16]	Ma Y. The mathematic magic of Photoshop blend modes for image processing[C]// 2011 International Conference on Multimedia Technology.IEEE, 2011:5159-5161.
[17]	Ren W, Pan J, Zhang H, et al. Single image dehazing via multi-scale convolutional neural networks with holistic edges[J]. International Journal of Computer Vision, 2020, 128(1):240-259. doi: 10.1007/s11263-019-01235-8
[18]	Dhara S K, Roy M, Sen D, et al. Color cast dependent image dehazing via adaptive airlight refinement and non-linear color balancing[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(5):2076-2081. doi: 10.1109/TCSVT.2020.3007850 url: https://ieeexplore.ieee.org/document/9134933/
[19]	Cho Y, Jeong J, Kim A. Model-assisted multiband fusion for single image enhancement and applications to robot vision[J]. IEEE Robotics and Automation Letters, 2018, 3(4):2822-2829. doi: 10.1109/ROBOT.1996.506590 url: http://ieeexplore.ieee.org/document/506590/
[20]	蒋刚毅, 黄大江, 王旭, 等. 图像质量评价方法研究进展[J]. 电子与信息学报, 2010, 32(1):219-226.
[20]	Jiang G Y, Huang D J, Wang X, et al. Overview on image quality assessment methods[J]. Journal of Electronics and Information Technology, 2010, 32(1):219-226.
[21]	Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12):4695-4708. doi: 10.1109/TIP.2012.2214050 pmid: 22910118

[1]	WANG Jianqiang, ZOU Zhaohui, LIU Rongbo, LIU Zhisong. A method for extracting information on coastal aquacultural ponds from remote sensing images based on a U²-Net deep learning model[J]. Remote Sensing for Natural Resources, 2023, 35(3): 17-24.
[2]	TANG Hui, ZOU Juan, YIN Xianghong, YU Shuchen, HE Qiuhua, ZHAO Dong, ZOU Cong, LUO Jianqiang. River and lake sand mining in the Dongting Lake area: Supervision based on high-resolution remote sensing images and typical case analysis[J]. Remote Sensing for Natural Resources, 2023, 35(3): 302-309.
[3]	XU Xinyu, LI Xiaojun, ZHAO Heting, GAI Junfei. Pansharpening algorithm of remote sensing images based on NSCT and PCNN[J]. Remote Sensing for Natural Resources, 2023, 35(3): 64-70.
[4]	LIU Li, DONG Xianmin, LIU Juan. A performance evaluation method for semantic segmentation models of remote sensing images considering surface features[J]. Remote Sensing for Natural Resources, 2023, 35(3): 80-87.
[5]	ZHANG Xian, LI Wei, CHEN Li, YANG Zhaoying, DOU Baocheng, LI Yu, CHEN Haomin. Research progress and prospect of remote sensing-based feature extraction of opencast mining areas[J]. Remote Sensing for Natural Resources, 2023, 35(2): 25-33.
[6]	DIAO Mingguang, LIU Yong, GUO Ningbo, LI Wenji, JIANG Jikang, WANG Yunxiao. Mask R-CNN-based intelligent identification of sparse woods from remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 97-104.
[7]	ZHENG Zongsheng, LIU Haixia, WANG Zhenhua, LU Peng, SHEN Xukun, TANG Pengfei. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 105-111.
[8]	XIONG Dongyang, ZHANG Lin, LI Guoqing. MaxEnt-based multi-class classification of land use in remote sensing image interpretation[J]. Remote Sensing for Natural Resources, 2023, 35(2): 140-148.
[9]	QIU Lei, ZHANG Xuezhi, HAO Dawei. VideoSAR moving target detection and tracking algorithm based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(2): 157-166.
[10]	ZHANG Ke, ZHANG Gengsheng, WANG Ning, WEN Jing, LI Yu, YANG Jun. A forecasting method for water table depths in areas with power transmission lines based on remote sensing and deep learning models[J]. Remote Sensing for Natural Resources, 2023, 35(1): 213-221.
[11]	JIN Yuanhang, XU Maolin, ZHENG Jiayuan. A dead tree detection algorithm based on improved YOLOv4-tiny for UAV images[J]. Remote Sensing for Natural Resources, 2023, 35(1): 90-98.
[12]	HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[13]	ZHAO Linghu, YUAN Xiping, GAN Shu, HU Lin, QIU Mingyu. An information extraction model of roads from high-resolution remote sensing images based on improved Deeplabv3+[J]. Remote Sensing for Natural Resources, 2023, 35(1): 107-114.
[14]	SHEN Jun’ao, MA Mengting, SONG Zhiyuan, LIU Tingzhou, ZHANG Wei. Water information extraction from high-resolution remote sensing images using the deep-learning based semantic segmentation model[J]. Remote Sensing for Natural Resources, 2022, 34(4): 129-135.
[15]	LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(4): 22-32.

Viewed

Full text

Abstract

Cited

Shared

Discussed