Please wait a minute...
 
国土资源遥感  2020, Vol. 32 Issue (3): 15-22    DOI: 10.6046/gtzyyg.2020.03.03
  技术方法 本期目录 | 过刊浏览 | 高级检索 |
深度卷积融合条件随机场的遥感图像语义分割
李宇1(), 肖春姣1, 张洪群1(), 李湘眷2, 陈俊1
1.中国科学院遥感与数字地球研究所,北京 100094
2.西安石油大学计算机学院,西安 710065
Remote sensing image semantic segmentation using deep fusion convolutional networks and conditional random field
LI Yu1(), XIAO Chunjiao1, ZHANG Hongqun1(), LI Xiangjuan2, CHEN Jun1
1. Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100094, China
2. School of Computer Science, Xi’an Shiyou University, Xi’an 710065, China
全文: PDF(2927 KB)   HTML  
输出: BibTeX | EndNote (RIS)      
摘要 

为了实现高分辨率光学遥感图像的语义分割,提出了一种基于深度卷积融合条件随机场的图像语义分割方法。该方法在全卷积神经网络模型的基础上增加反卷积融合结构结合不同深度的池化层结果,将浅层的细节信息和高层的语义信息融入网络模型,同时将条件随机场的参数推断以迭代层的形式嵌入网络架构,搭建网络模型,在模型训练的正反向传播过程中综合利用遥感图像丰富的细节信息和上下文信息,实现端到端的遥感图像语义分割。在高分辨率遥感图像中进行的实验结果显示: 随着模型中反卷积融合结构结合池化层深度的增加,语义分割处理精度逐渐提高,语义分割结果中的地物轮廓也更清晰、准确; 上下文信息的引入也在一定程度上提高了图像语义分割的精度。实验表明该方法能够较好地保持语义对象内部区域的一致性,有效提高图像语义分割的精度。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
李宇
肖春姣
张洪群
李湘眷
陈俊
关键词 遥感图像语义分割全卷积神经网络条件随机场    
Abstract

A method for remote sensing image semantic segmentation based on deep fusion convolutional networks and conditional random field is proposed. First, the fully convolutional networks framework with deconvolutional fusion structure is utilized to integrate the pooling-layer results at different levels. The low-level features with rich detail information are combined with the high-level features via deconvolutional fusion module. At the same time, the parameter inference process of conditional random field is embedded in the network architecture by adding recurrent neural networks iteration layers. In addition, the deep fusion convolutional networks and conditional random field model is established. Then, in the model training stage, the abundant detail information and context information in the image are introduced simultaneously to the positive and negative propagation. And lastly, the remote sensing image semantic segmentation is accomplished by the end-to-end framework. Semantic segmentation experiments were performed on the high-resolution optical remote sensing images, and the results show that, with the increase of the depth of deconvolution fusion layer in the model, semantic segmentation results are more refined, and the contour of terrestrial object is more accurate. The introduction of context information also improves the accuracy of image semantic segmentation to a certain extent. It is concluded that the proposed method can better maintain the consistency of internal areas of semantic object and effectively improve the accuracy of semantic segmentation.

Key wordsremote sensing image    semantic segmentation    fully convolutional networks    conditional random field
收稿日期: 2019-08-16      出版日期: 2020-10-09
:  TP751  
基金资助:国家重点研发计划项目“天空地一体化协同观测、数据整合与应急信息提取技术研究”(2016YFB0502502);国家自然科学基金项目“基于语义模型的高分辨率卫星遥感图像人造目标检测方法研究”(61501460)
通讯作者: 张洪群
作者简介: 李 宇(1986-),女,博士,工程师,主要从事遥感图像处理的研究。Email: liyu@radi.ac.cn
引用本文:   
李宇, 肖春姣, 张洪群, 李湘眷, 陈俊. 深度卷积融合条件随机场的遥感图像语义分割[J]. 国土资源遥感, 2020, 32(3): 15-22.
LI Yu, XIAO Chunjiao, ZHANG Hongqun, LI Xiangjuan, CHEN Jun. Remote sensing image semantic segmentation using deep fusion convolutional networks and conditional random field. Remote Sensing for Land & Resources, 2020, 32(3): 15-22.
链接本文:  
https://www.gtzyyg.com/CN/10.6046/gtzyyg.2020.03.03      或      https://www.gtzyyg.com/CN/Y2020/V32/I3/15
Fig.1  图像语义分割处理流程
Fig.2  DFN-CRF网络模型
模型 训练时间/min 测试时间/s
DFN-CRF-5 474 0.61
DFN-CRF-4 5 013 0.69
DFN-CRF-3 5 237 0.73
DFN-CRF-2 5 589 0.74
DFN-CRF-1 5 976 0.77
Tab.1  各模型训练及测试时间
Tab.2  采用不同层级融合结构的DFN-CRF模型语义分割结果对比
模型 PA CA MIOU
DFN-CRF-5 0.908 990 0.855 277 0.768 730
DFN-CRF-4 0.910 455 0.878 851 0.799 701
DFN-CRF-3 0.918 773 0.897 060 0.809 149
DFN-CRF-2 0.918 969 0.898 627 0.816 557
DFN-CRF-1 0.919 967 0.900 086 0.819 535
Tab.3  各模型语义分割结果定量评价
模型 训练时间/min PA CA MIOU
DFN-1 5 749 0.912 465 0.883 090 0.811 262
DFN-CRF-1 5 976 0.919 967 0.900 086 0.819 535
Tab.4  DFN-1和DFN-CRF-1语义分割结果定量评价
Fig.3  DFN-1和DFN-CRF-1地物类别PA对比
模型 训练时间/min MIOU
DeepLab 1 692 0.783 401
CRFasRNN 4 853 0.798 024
FCN8s 4 917 0.805 267
DFN-CRF-1 5 976 0.819 535
Tab.5  本文方法与其他方法结果对比
Fig.4-1  UC Merced Land-Use数据集实验结果
Fig.4-2  UC Merced Land-Use数据集实验结果
[1] 魏云超, 赵耀. 基于DCNN的图像语义分割综述[J]. 北京交通大学学报, 2016,40(4):82-91.
Wei Y C, Zhao Y. A review on image semantic segmentation based on DCNN[J]. Journal of Beijing Jiaotong University, 2016,40(4):82-91.
[2] 罗冰. 语义对象分割方法研究[D]. 成都:电子科技大学, 2018.
Luo B. Research on segmentation of semantic objects[D]. Chengdu:University of Electronic Science and Technology of China, 2018.
[3] 韩铮, 肖志涛. 基于纹元森林和显著性先验的弱监督图像语义分割方法[J]. 电子与信息学报, 2018,40(3):610-617.
Han Z, Xiao Z T. Weakly supervised semantic segmentation based on semantic texton forest and saliency prior[J]. Journal of Electronics & Information Technology, 2018,40(3):610-617.
[4] Zhao J, Zhong Y F, Zhang L P. Detail-preserving smoothing classifier based on conditional random fields for high spatial resolution remote sensing imagery[J]. IEEE Transactions on Geoscience & Remote Sensing, 2015,53(5):2440-2452.
[5] 叶发茂, 罗威, 苏燕飞, 等. 卷积神经网络特征在遥感图像配准中的应用[J]. 国土资源遥感, 2019,31(2):32-37.doi: 10.6046/gtzyyg.2019.02.05.
Ye F M, Luo W, Su Y F, et al. Application of convolutional neural network feature to remote sensing image registration[J]. Remote Sensing for Land and Resources, 2019,31(2):32-37.doi: 10.6046/gtzyyg.2019.02.05.
[6] Zhao W D, Li S S, Li A, et al. Hyperspectral images classification with convolutional neural network and textural feature using limited training samples[J]. Remote Sensing Letters, 2019,10(5):449-458.
doi: 10.1080/2150704X.2019.1569274
[7] 张康, 黑保琴, 李盛阳, 等. 基于CNN模型的遥感图像复杂场景分类[J]. 国土资源遥感, 2018,30(4):49-55.doi: 10.6046/gtzyyg.2018.04.08.
Zhang K, Hei B Q, Li S Y, et al. Complex scene classification of remote sensing images based on CNN[J]. Remote Sensing for Land and Resources, 2018,30(4):49-55.doi: 10.6046/gtzyyg.2018.04.08.
[8] Zhang R, Li G Y, Li M L, et al. Fusion of images and point clouds for the semantic segmentation of large-scale 3D scenes based on deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,143:85-96.
doi: 10.1016/j.isprsjprs.2018.04.022
[9] Kanmffmeyer M, Salberg A B, Jenssen R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks [C]//Proceedings of Computer Vision and Pattern Recognition Workshops. 2016: 680-688.
[10] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2015: 3431-3440.
[11] Sherrah J. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery[EB/OL].(2016-06-08)[2019-08-16]. http://arxiv.org/abs/1606.02585v1.
[12] Badrinarayanan V, Handa A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,309(12):2481-2495.
[13] Ronneberger O, Fischer P, Brox T. U-Net:Convolutional networks for biomedical image segmentation [C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention.Berlin:Springer, 2015: 234-241.
[14] Chen L C, Papandreou G, Kokkinos L, et al. DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,40(4):834-848.
doi: 10.1109/TPAMI.2017.2699184 pmid: 28463186
[15] Zheng S, Jayasumana S, Romera-Paredes B, et al. Conditional random fields as recurrent neural networks [C]//IEEE International Conference on Computer Vision. 2015: 1529-1537.
[16] Lafferty J, Mccallum A, Pereira F. Conditional random fields:Probabilistic models for segmenting and labeling sequence data [C]//Eighteenth International Conference on Machine Learning. 2001,3(2):282-289.
[17] Hammersley J M, Clifford P. Markov fields on finite graphs and lattices[EB/OL].[ 2012- 01- 30]. http://www.statslab.cam.ac.uk/~grg/books/hammfest/hamm-cliff.pdf.
[18] Krahenbuhl P, Koltun V. Efficient inference in fully connected CRFs with gaussian edge potentials [C]//Proceeding NIPS’11 Proceeding of the 24th International Conference on Neural Information Processing Systems Advances in Neural Information Processing Systems. 2011: 109-117.
[19] Cheng G, Han J, Lu X Q. Remote sensing image scene classification:benchmark and state of the art [C]//Proceedings of the IEEE. 2017,105(10):1865-1883.
[20] Yang Y, Newsam S. Bag-of-visual-words and spatial extensions for land-use classification [C]//Proceedings of the 18th Sigspatial International Conference on Advances in Geographic Information Systems. 2010: 270-279.
[21] Alberto G G, Sergio O E, Sergiu O, et al. A review on deep learning techniques applied to semantic segmentation[EB/OL].(2017-04-22)[2019-08-16]. http://arxiv.org/abs/1704.06857.
[1] 张大明, 张学勇, 李璐, 刘华勇. 一种超像素上Parzen窗密度估计的遥感图像分割方法[J]. 自然资源遥感, 2022, 34(1): 53-60.
[2] 李轶鲲, 杨洋, 杨树文, 王子浩. 耦合模糊C均值聚类和贝叶斯网络的遥感影像后验概率空间变化向量分析[J]. 自然资源遥感, 2021, 33(4): 82-88.
[3] 刘万军, 高健康, 曲海成, 姜文涛. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3): 97-106.
[4] 郭文, 张荞. 基于注意力增强全卷积神经网络的高分卫星影像建筑物提取[J]. 国土资源遥感, 2021, 33(2): 100-107.
[5] 刘钊, 赵桐, 廖斐凡, 李帅, 李海洋. 基于语义分割网络的高分遥感影像城市建成区提取方法研究与对比分析[J]. 国土资源遥感, 2021, 33(1): 45-53.
[6] 蔡祥, 李琦, 罗言, 齐建东. 面向对象结合深度学习方法的矿区地物提取[J]. 国土资源遥感, 2021, 33(1): 63-71.
[7] 仇一帆, 柴登峰. 无人工标注数据的Landsat影像云检测深度学习方法[J]. 国土资源遥感, 2021, 33(1): 102-107.
[8] 王小兵. 融合提升小波阈值与多方向边缘检测的矿区遥感图像去噪[J]. 国土资源遥感, 2020, 32(4): 46-52.
[9] 刘钊, 廖斐凡, 赵桐. 基于PSPNet的遥感影像城市建成区提取及其优化方法[J]. 国土资源遥感, 2020, 32(4): 84-89.
[10] 刘尚旺, 崔智勇, 李道义. 基于Unet网络多任务学习的遥感图像建筑地物语义分割[J]. 国土资源遥感, 2020, 32(4): 74-83.
[11] 蔡之灵, 翁谦, 叶少珍, 简彩仁. 基于Inception-V3模型的高分遥感影像场景分类[J]. 国土资源遥感, 2020, 32(3): 80-89.
[12] 刘文雅, 岳安志, 季珏, 师卫华, 邓孺孺, 梁业恒, 熊龙海. 基于DeepLabv3+语义分割模型的GF-2影像城市绿地提取[J]. 国土资源遥感, 2020, 32(2): 120-129.
[13] 于博, 张军军, 李春庚, 安居白. 图像语义分割辅助的车载激光点云道路提取方法[J]. 国土资源遥感, 2020, 32(1): 66-74.
[14] 叶发茂, 罗威, 苏燕飞, 赵旭青, 肖慧, 闵卫东. 卷积神经网络特征在遥感图像配准中的应用[J]. 国土资源遥感, 2019, 31(2): 32-37.
[15] 谢奇芳, 姚国清, 张猛. 基于Faster R-CNN的高分辨率图像目标检测技术[J]. 国土资源遥感, 2019, 31(2): 38-43.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
京ICP备05055290号-2
版权所有 © 2015 《自然资源遥感》编辑部
地址:北京学院路31号中国国土资源航空物探遥感中心 邮编:100083
电话:010-62060291/62060292 E-mail:zrzyyg@163.com
本系统由北京玛格泰克科技发展有限公司设计开发