A performance evaluation method for semantic segmentation models of remote sensing images considering surface features

doi:10.6046/zrzyyg.2022164

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(4774 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Deep semantic segmentation has been widely applied in land monitoring and interpretation based on remote sensing images. However, existing quality evaluation methods cannot reflect the preserved spatial geometric features of semantic segmentation results. Based on the practical demand for remote sensing image interpretation, surveying, and mapping, this study proposed a method for the performance evaluation of semantic segmentation models for remote sensing images considering geoscience features: the connectivity similarity index (CSIM). From the perspective of the connectivity similarity of surface feature spots in remote sensing images, the CSIM method embedded the surface features into the performance evaluation system of semantic segmentation models. The CSIM method allows for quantitatively evaluating the connectivity similarity of spots between the semantic segmentation results of remote sensing images and the actual sample labels, thus accurately describing the preserved spot integrity in the predicted classification results. Therefore, the CSIM method can objectively determine the applicability of a pre-training model to remote sensing image interpretation in surveying and mapping production. As substantiated by a lot of practice, the CSIM method can monitor and control the model training in real time, effectively guide the selection of the optimal pre-training model, and accurately evaluate the quality of remote sensing image interpretation results considering geoscience features. Therefore, the CSIM method is critical for deep-learning-enabled remote sensing image interpretation, surveying, and mapping.

Keywords model performance evaluation semantic segmentation geoscience feature remote sensing interpretation deep learning

ZTFLH:

TP79

Issue Date: 19 September 2023

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Li LIU
	Xianmin DONG
	Juan LIU

Cite this article:

Li LIU,Xianmin DONG,Juan LIU. A performance evaluation method for semantic segmentation models of remote sensing images considering surface features[J]. Remote Sensing for Natural Resources, 2023, 35(3): 80-87.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2022164 OR https://www.gtzyyg.com/EN/Y2023/V35/I3/80

Tab.1 Common evaluation indicator

Fig.1 Comparison chart between real label and predicted labels

Tab.2 Evaluation results of predicted labels

Fig.2 Comparison of model performance evaluation methods

Fig.3 Connected map spots

Fig.4 Spatial intersection relation

Fig.5 Spatial calibration diagram

Fig.6 Dynamic connected area regularity

Fig.7 Test image and real label

Tab.3 Experimental parameters of model training

Fig.8 Evolution of model training evaluation indicator

Fig.9 Comparison of connected map spots in predicted label by models in different periods

Fig.10 CSIM evaluation value of target model set

Fig.11 Predicted label of local area of test image in different periods

[1]	龚健雅, 许越, 胡翔云, 等. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8):1013-1022. doi: 10.11947/j.AGCS.2021.20210085
[1]	Gong J Y, Xu Y, Hu X Y, et al. Status analysis and research of sample database for intelligent interpretation of remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1013-1022. doi: 10.11947/j.AGCS.2021.20210085
[2]	Rawat W, Wang Z H. Deep convolutional neural networks for image classifycation:A comprehensive review[J]. Neural Computation, 2017, 29(9):2352-2449. doi: 10.1162/neco_a_00990 url: https://direct.mit.edu/neco/article/29/9/2352-2449/8292
[3]	Yuan Q Q, Shen H F, Li T W, et al. Deep learning in environmental remote sensing:Achievements and challenges[J]. Remote Sensing of Environment, 2020, 241:111716. doi: 10.1016/j.rse.2020.111716 url: https://linkinghub.elsevier.com/retrieve/pii/S0034425720300857
[4]	Zhang L P, Zhang L F, Du B. Deep learning for remote sensing data:A technical tutorial on the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2016, 4(2):22-40. doi: 10.1109/MGRS.2016.2540798 url: http://ieeexplore.ieee.org/document/7486259/
[5]	洪亮, 冯亚飞, 彭双云, 等. 面向对象的多尺度加权联合稀疏表示的高空间分辨率遥感影像分类[J]. 测绘学报, 2022, 51(2):224-237. doi: 10.11947/j.AGCS.2022.20190290
[5]	Hong L, Feng Y F, Peng S Y, et al. Classification of high spatial resolution remote sensing imagery based on object-oriented multiscale weighted sparse representation[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(2):224-237.
[6]	Zhang P L, Lyu Z Y, Shi W Z. Object-based spatial feature for classification of very high resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 10(6):1572-1576. doi: 10.1109/LGRS.2013.2262132 url: http://ieeexplore.ieee.org/document/6573351/
[7]	段嘉鑫. 基于上下文信息的图像分割结果质量评价方法研究[D]. 成都: 电子科技大学, 2021.
[7]	Duan J X. Research on evaluation method of image segmentation results quality based on context information[D]. Chengdu: University of Electronic Science and Technology, 2021.
[8]	党宇. 基于卷积神经网络的地表覆盖分类自动质量评价方法研究[D]. 徐州: 中国矿业大学, 2018.
[8]	Dang Y. Study on automatic evaluation method of land cover classification based on convolutional neural network[D]. Xuzhou: University of Mining and Technology of China, 2018.
[9]	郑明国, 蔡强国, 秦明周, 等. 一种遥感影像分类精度检验的新方法[J]. 遥感学报, 2006, 10(1):39-48.
[9]	Zheng M G, Cai Q G, Qin M Z, et al. A new approach to accuracy assessment of classifications of remotely sensed data[J]. Journal of Remote Sensing, 2006, 10(1):39-48.
[10]	欧阳松. 地学知识嵌入的遥感影像深度语义分割方法研究[D]. 武汉: 武汉大学, 2021.
[10]	Ouyang S. The methodology research on geographic knowledge-guided deep semantic segmentation of remote sensing imagery[D]. Wuhan: Wuhan University, 2021.
[11]	陶超, 阴紫薇, 朱庆, 等. 遥感影像智能解译:从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8):1122-1134. doi: 10.11947/j.AGCS.2021.20210089
[11]	Tao C, Yin Z W, Zhu Q, et al. Remote sensing image intelligent interpretation:From supervised learning to self-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1122-1134.
[12]	龚健雅. 人工智能时代测绘遥感技术的发展机遇与挑战[J]. 武汉大学学报(信息科学版), 2018, 43(12):1788-1796.
[12]	Gong J Y. Chances and challenges for development of surveying and remote sensing in the age of artificial intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1788-1796.
[13]	叶利华, 王磊, 张文文, 等. 高分辨率光学遥感场景分类的深度度量学习方法[J]. 测绘学报, 2019, 48(6):698-707. doi: 10.11947/j.AGCS.2019.20180434
[13]	Ye L H, Wang L, Zhang W W, et al. Deep metric learning method for high resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):698-707. doi: 10.11947/j.AGCS.2019.20180434
[14]	常亮, 邓小明, 周明全, 等. 图像理解中的卷积神经网络[J]. 自动化学报, 2016, 42(9):1300-1312.
[14]	Chang L, Deng X M, Zhou M Q, et al. Convolutional neural networks in image understanding[J]. Acta Automatica Sinica, 2016, 42(9): 1300-1312.
[15]	曹玉东, 刘海燕, 曹旭, 等. 基于深度学习的图像质量评价方法综述[J]. 计算机工程与应用, 2021, 57(23):27-36. doi: 10.3778/j.issn.1002-8331.2106-0228
[15]	Cao Y D, Liu H Y. Cao X, et al. Overview of image quality assessment method based on deep learning[J]. Computer Engineering and Applications, 2021, 57(23):27-36. doi: 10.3778/j.issn.1002-8331.2106-0228
[16]	钱晓亮, 李佳, 程塨, 等. 特征提取策略对高分辨率遥感图像场景分类性能影响的评估[J]. 遥感学报, 2018, 22(5):758-776.
[16]	Qian X L, Li J, Cheng G, et al. Evaluation of the effect of feature extraction strategy on the performance of high-resolution remote sensing image scene classification[J]. Journal of Remote Sensing, 2018, 22(5):758-776.
[17]	李琳, 余胜生. 基于深度学习模型的图像质量评价方法[J]. 华中科技大学学报(自然科学版), 2016, 44(12):70-75.
[17]	Li L, Yu S S. Image quality evaluation method based on deep learning model[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2016, 44(12):70-75.
[18]	薛廉, 周春兰. 基于监督分类分区域的特征因子提取[J]. 地理空间信息, 2009, 7(2):100-102.
[18]	Xue L, Zhou C L. Feature factor extraction based on supervised classification subregions[J]. Geospatial Information, 2009, 7(2):100-102.

[1]	NIU Xianghua, HUANG Wei, HUANG Rui, JIANG Sili. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.
[2]	LIN Jiahui, LIU Guang, FAN Jinghui, ZHAO Hongli, BAI Shibiao, PAN Hongyu. Extracting information about mining subsidence by combining an improved U-Net model and D-InSAR[J]. Remote Sensing for Natural Resources, 2023, 35(3): 145-152.
[3]	ZHANG Xian, LI Wei, CHEN Li, YANG Zhaoying, DOU Baocheng, LI Yu, CHEN Haomin. Research progress and prospect of remote sensing-based feature extraction of opencast mining areas[J]. Remote Sensing for Natural Resources, 2023, 35(2): 25-33.
[4]	DIAO Mingguang, LIU Yong, GUO Ningbo, LI Wenji, JIANG Jikang, WANG Yunxiao. Mask R-CNN-based intelligent identification of sparse woods from remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 97-104.
[5]	QIU Lei, ZHANG Xuezhi, HAO Dawei. VideoSAR moving target detection and tracking algorithm based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(2): 157-166.
[6]	ZHANG Ke, ZHANG Gengsheng, WANG Ning, WEN Jing, LI Yu, YANG Jun. A forecasting method for water table depths in areas with power transmission lines based on remote sensing and deep learning models[J]. Remote Sensing for Natural Resources, 2023, 35(1): 213-221.
[7]	HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[8]	ZHAO Linghu, YUAN Xiping, GAN Shu, HU Lin, QIU Mingyu. An information extraction model of roads from high-resolution remote sensing images based on improved Deeplabv3+[J]. Remote Sensing for Natural Resources, 2023, 35(1): 107-114.
[9]	MENG Congtang, ZHAO Yindi, HAN Wenquan, HE Chenyang, CHEN Xiqiu. RandLA-Net-based detection of urban building change using airborne LiDAR point clouds[J]. Remote Sensing for Natural Resources, 2022, 34(4): 113-121.
[10]	SHEN Jun’ao, MA Mengting, SONG Zhiyuan, LIU Tingzhou, ZHANG Wei. Water information extraction from high-resolution remote sensing images using the deep-learning based semantic segmentation model[J]. Remote Sensing for Natural Resources, 2022, 34(4): 129-135.
[11]	LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(4): 22-32.
[12]	TAN Hai, ZHANG Rongjun, FAN Wenfeng, ZHANG Yifang, XU Hang. Classification and detection of radiation anomalies in Chinese optical satellite images by integrating multi-scale features[J]. Remote Sensing for Natural Resources, 2022, 34(4): 97-104.
[13]	SU Wei, LIN Yangyang, YUE Wen, CHEN Yingbiao. Identification of mariculture areas in Guangdong Province and remote sensing monitoring of their spatial and temporal changes based on the U-Net convolutional neural network[J]. Remote Sensing for Natural Resources, 2022, 34(4): 33-41.
[14]	ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
[15]	ERPAN Anwar, MAMAT Sawut, MAIHEMUTI Balati. Recognition of cotton distribution based on GF-2 images and Unet model[J]. Remote Sensing for Natural Resources, 2022, 34(2): 242-250.

Viewed

Full text

Abstract

Cited

Shared

Discussed