Please wait a minute...
 
Remote Sensing for Natural Resources    2024, Vol. 36 Issue (3) : 137-145     DOI: 10.6046/zrzyyg.2023109
|
Improved Transformer-based hyperspectral image classification method for surface features: A case study of the Yellow River Delta
LI Wei1(), FAN Yanguo1(), ZHOU Peixi2
1. College of Oceanography and Space Informatics,China University of Petroleum(East China),Qingdao 266580, China
2. Qingdao Hongyi Tiantu Information Technology Co., Ltd., Qingdao 266555, China
Download: PDF(10380 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Hyperspectral technology has become the major means of coastal wetland monitoring. However, traditional hyperspectral classification methods usually face challenges such as insufficient feature extraction, the same surface features corresponding to different spectra, and fragmented scenes. To solve these problems, this study proposed a new classification method by applying Transformer to hyperspectral classification. This vision Transformer (ViT)-based method expanded the receptive field by learning global spatial features using non-local technology, thus overcoming the insufficient extraction of discriminant features. Meanwhile, this method enhanced the cross-layer information interchange through cross-layer adaptive residual connection, thus eliminating information loss. This study, taking NC16 and NC13 wetland datasets of the Yellow River Delta as experimental data, compared the classification method proposed in this study to support vector machine (SVM), one-dimensional convolution neural network (1DCNN), contextual deep convolution neural network (CDCNN), spectral-spatial residual network (SSRN), hybrid spectral network (HybridSN), and ViT. The comparison results show that the new method yielded significantly elevated overall accuracy (OA) of up to 96.24% and 73.84%, average accuracy (AA) reaching 83.42% and 74.87%, and Kappa coefficients of up to 94.80% and 68.94%, respectively for the two datasets.
Keywords hyperspectral      wetland classification      Transformer      non-local spatial feature     
ZTFLH:  TP79  
  P237  
Issue Date: 03 September 2024
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Wei LI
Yanguo FAN
Peixi ZHOU
Cite this article:   
Wei LI,Yanguo FAN,Peixi ZHOU. Improved Transformer-based hyperspectral image classification method for surface features: A case study of the Yellow River Delta[J]. Remote Sensing for Natural Resources, 2024, 36(3): 137-145.
URL:  
https://www.gtzyyg.com/EN/10.6046/zrzyyg.2023109     OR     https://www.gtzyyg.com/EN/Y2024/V36/I3/137
Fig.1  This article improves the algorithm model diagram
Fig.2  Non-local module and implementation details
Fig.3  Adaptive cross-layer residual connection module
Fig.4  Location of the study area
类别 查全率
SVM 1DCNN CDCNN SSRN HybridSN ViT 本文方法
碱蓬 93.91 99.76 99.63 99.64 96.23 98.78 99.87
水泥路 94.46 88.02 92.19 87.61 78.10 64.34 81.59
沥青柏油路 93.12 87.09 87.02 87.97 74.33 80.28 91.56
水域 89.97 99.10 91.67 98.38 96.41 97.75 99.99
石块 71.45 93.19 95.15 93.40 76.00 95.48 78.23
草地 80.52 76.74 73.33 73.71 91.53 74.47 74.44
铁杆 0 0 0 12.50 0 31.25 33.50
柽柳 0 60.46 85.98 55.81 49.87 54.56 82.29
枯萎的芦苇 50.16 57.40 66.24 53.03 79.66 70.28 66.25
芦苇 47.15 52.76 78.23 73.18 54.18 63.69 71.19
互花米草 98.72 94.11 95.36 86.54 92.00 95.74 96.37
苔藓 31.50 67.89 71.50 76.42 98.02 73.72 72.16
旱田 84.93 82.00 88.64 92.74 96.57 96.07 94.94
湿地 97.36 91.92 91.68 94.64 99.93 94.15 97.40
滩涂 65.41 82.19 93.91 68.20 93.91 88.01 94.87
标准反射板 0 100.00 57.60 43.80 31.00 93.75 100.00
OA 85.76 92.63 92.72 94.99 94.18 94.62 96.24
AA 62.42 77.04 79.25 74.85 75.48 79.52 83.42
Kappa 80.82 89.86 90.01 93.12 92.11 92.62 94.80
Tab.1  Classification results of different methods for NC16 datasets(%)
Fig.5  NC16 dataset classification result plot
类别名称 查全率
SVM 1DCNN CDCNN SSRN HybridSN ViT 本文方法
碱蓬 86.08 80.37 80.97 91.10 95.53 88.30 92.23
沥青水泥混合路面 99.96 99.77 98.64 99.37 100.00 94.05 99.73
湿地 77.10 87.17 86.65 90.04 70.77 73.38 80.37
水域 99.98 99.97 98.82 99.81 100.00 99.54 99.76
石油 90.35 93.78 91.08 99.54 95.20 94.24 99.04
芦苇 31.84 46.96 53.91 44.27 52.84 52.00 50.89
柽柳 0 38.94 60.00 50.56 17.04 46.36 57.27
车 27.36 65.49 77.74 87.72 72.93 57.36 85.17
旱田 53.67 81.13 87.64 66.06 98.27 92.00 74.97
标准反射板 0 100.00 37.65 97.37 0 89.79 51.02
柽柳芦苇混生 35.12 56.63 53.44 59.60 59.13 51.73 52.13
碱蓬芦苇混生 61.56 64.79 69.23 46.54 69.73 51.98 65.12
芦苇水域混合 45.82 49.52 49.88 65.77 58.54 68.72 65.61
OA 69.38 70.41 69.81 71.46 70.65 70.86 73.84
AA 54.53 74.19 72.74 76.75 68.46 73.80 74.87
Kappa 62.73 64.91 64.29 66.18 65.19 65.47 68.94
Tab.2  Classification results of different methods for NC13 datasets (%)
Fig.6  NC13 dataset classification result plot
Non-local CAF NC16 NC13
OA AA Kappa OA AA Kappa
× × 94.62 79.52 92.62 70.86 73.80 65.47
× √ 95.41 76.82 93.67 71.26 74.49 65.94
√ × 95.61 79.97 93.94 71.73 74.72 66.51
√ √ 96.24 83.42 94.80 73.84 74.87 68.94
Tab.3  Results of ablation experiments(%)
[1] Xia Y, Fang C, Lin H, et al. Spatiotemporal evolution of wetland eco-hydrological connectivity in the Poyang Lake area based on long time-series remote sensing images[J]. Remote Sensing, 2021, 13(23):4812.
[2] López-Tapia S, Ruiz P, Smith M, et al. Machine learning with high-resolution aerial imagery and data fusion to improve and automate the detection of wetlands[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 105:102581.
[3] Sun S, Wang Y, Song Z, et al. Modelling aboveground biomass carbon stock of the Bohai rim coastal wetlands by integrating remote sensing,terrain,and climate data[J]. Remote Sensing, 2021, 13(21):4321.
[4] 王欣, 樊彦国. 基于改进DenseNet和空谱注意力机制的高光谱图像分类[J]. 激光与光电子学进展, 2022, 59(2):0210014.
[4] Wang X, Fan Y G. Hyperspectral image classification based on modified DenseNet and spatial spectrum attention mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(2):0210014.
[5] 郑宗生, 刘海霞, 王振华, 等. 改进3D-CNN的高光谱图像地物分类方法[J]. 自然资源遥感, 2023, 35(2):105-111.doi:10.6046/zrzyyg.2022100.
[5] Zheng Z S, Liu H X, Wang Z H, et al. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2):105-111.doi:10.6046/zrzyyg.2022100.
[6] Duan P, Kang X, Li S, et al. Multichannel pulse-coupled neural network-based hyperspectral image visualization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2444-2456.
[7] 贾少鹏, 高红菊, 杭潇. 基于深度学习的农作物病虫害图像识别技术研究进展[J]. 农业机械学报, 2019, 50(s1):313-317.
[7] Jia S P, Gao H J, Hang X. Research progress on image recognition technology of crop pests and diseases based on deep learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(s1):313-317.
[8] Chen Y, Lin Z, Zhao X, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2094-2107.
[9] Mou L, Ghamisi P, Zhu X X. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[10] Paoletti M E, Haut J M, Fernandez-Beltran R, et al. Capsule networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(4):2145-2160.
[11] Zhu L, Chen Y, Ghamisi P, et al. Generative adversarial networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9):5046-5063.
[12] 陈禹汗, 王波, 严清赟, 等. 结合Transformer与多尺度残差机制的高光谱遥感分类[J]. 激光与光电子学进展, 2023, 60(12):3788/LOP220921.
[12] Chen Y H, Wang B, Yan Q Y, et al. Hyperspectral remote-sensing classification combining transformer and multiscale residual mechanisms[J]. Laser & Optoelectronics Progress, 2023, 60(12):3788/LOP220921.
[13] Sun L, Zhao G, Zheng Y, et al. Spectral-spatial feature tokenization transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5522214.
[14] Hong D, Han Z, Yao J, et al. SpectralFormer:Rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5518615.
[15] Zhong Z, Li Y, Ma L, et al. Spectral-spatial transformer network for hyperspectral image classification:A factorized architecture search framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5514715.
[16] Liu K, Sun W, Shao Y, et al. Mapping coastal wetlands using transformer in transformer deep network on China ZY1-02D hyperspectral satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:3891-3903.
[17] Gao Y, Song X, Li W, et al. Fusion classification of HSI and MSI using a spatial-spectral vision transformer for wetland biodiversity estimation[J]. Remote Sensing, 2022, 14(4):850.
[18] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J/OL]. 2020.arXiv,2010.11929[2023-6-17]. https://arxiv.org/abs/2010.11929v1.
url: https://arxiv.org/abs/2010.11929v1
[19] Wang C, Bai X, Zhou L, et al. Hyperspectral image classification based on non-local neural networks[C]// IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium.Yokohama,Japan.IEEE, 2019: 584-587.
[20] Ma H, Liu G, Yuan Y. Enhanced non-local cascading network with attention mechanism for hyperspectral image denoising[C]// 2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Barcelona,Spain.IEEE, 2020:2448-2452.
[21] Lei R, Zhang C, Du S, et al. A non-local capsule neural network for hyperspectral remote sensing image classification[J]. Remote Sensing Letters, 2021, 12(1):40-49.
[22] He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks[C]// European Conference on Computer Vision.Cham:Springer, 2016:630-645.
[23] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,HI,USA.IEEE, 2017:2261-2269.
[24] Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer, 2015:234-241.
[25] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).June 27-30,2016.Las Vegas,NV,USA.IEEE, 2016:770-778.
[26] Hinton G E, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. Computer Science, 2015, 14(7):38-39.
[27] Xie Z, Hu J, Kang X, et al. Multilayer global spectral-spatial attention network for wetland hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5518913.
[28] Melgani F, Bruzzone L. Classification of hyperspectral remote sensing images with support vector machines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(8):1778-1790.
[29] Hu W, Huang Y, Wei L, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015:258619.
[30] Lee H, Kwon H. Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2017, 26(10):4843-4855.
[31] Zhong Z, Li J, Luo Z, et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):847-858.
[32] Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[33] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017,30.
[1] YI Zifang, ZHOU Leilei, LUO Jianlan, CAO Li. A method for hyperspectral inversion of element contents for soil-quality evaluation of cultivated land[J]. Remote Sensing for Natural Resources, 2024, 36(3): 225-232.
[2] SONG Qi, GAO Xiaohong, SONG Yuting, LI Qiaoli, CHEN Zhen, LI Runxiang, ZHANG Hao, CAI Sangjie. Estimation of soil organic carbon content in farmland based on UAV hyperspectral images: A case study of farmland in the Huangshui River basin[J]. Remote Sensing for Natural Resources, 2024, 36(2): 160-172.
[3] ZHAO Heting, LI Xiaojun, XU Xinyu, GAI Junfei. An ICM-based adaptive pansharpening algorithm for hyperspectral images[J]. Remote Sensing for Natural Resources, 2024, 36(2): 97-104.
[4] YU Hang, TAN Bingxiang, SHEN Mingtan, HE Chenrui, HUANG Yifei. Identifying predominant tree species based on airborne hyperspectral images using machine learning algorithms[J]. Remote Sensing for Natural Resources, 2024, 36(1): 118-127.
[5] CAO Delong, LIN Zhen, TANG Tingyuan, LI Chuyu, WANG Xiaorui. Application of 3D-HRNet in Zhuhai-1 OHS hyperspectral images for natural resource monitoring and eco-environment analysis[J]. Remote Sensing for Natural Resources, 2023, 35(4): 34-42.
[6] LI Lei, SUN Xiyan, JI Yuanfa, FU Wentao. Hyperspectral image classification based on superpixel segmentation and extended multi-attribute profiles[J]. Remote Sensing for Natural Resources, 2023, 35(4): 114-121.
[7] JIANG Zhuoran, ZHOU Xinxin, CAO Wei, WANG Yahua, WU Changbin. Intelligent detection of crab ponds using remote sensing images based on a cooperative interpretation mechanism[J]. Remote Sensing for Natural Resources, 2023, 35(3): 25-34.
[8] ZHENG Zongsheng, LIU Haixia, WANG Zhenhua, LU Peng, SHEN Xukun, TANG Pengfei. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 105-111.
[9] ZHANG Guojian, LIU Shengzhen, SUN Yingjun, YU Kaijie, LIU Lina. Hyperspectral anomaly detection based on the weakly supervised robust autoencoder[J]. Remote Sensing for Natural Resources, 2023, 35(2): 167-175.
[10] KONG Zhuo, YANG Haitao, ZHENG Fengjie, LI Yang, QI Ji, ZHU Qinyu, YANG Zhonglin. Research advances in atmospheric correction of hyperspectral remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(4): 1-10.
[11] ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
[12] SUN Xiao, XU Linlin, WANG Xiaoyang, TIAN Ye, WANG Wei, ZHANG Zhongyue. Mineral identification from hyperspectral images based on the optimized K-P-Means unmixing method[J]. Remote Sensing for Natural Resources, 2022, 34(3): 43-49.
[13] YAN Hongbo, WEI Wanqiu, LU Xianjian, YANG Zhigao, LI Zhenbao. A review of remote sensing inversion methods for estimating soil water content based on hyperspectral characteristics[J]. Remote Sensing for Natural Resources, 2022, 34(2): 1-9.
[14] SUN Xiao, PENG Junhuan, ZHAO Feng, WANG Xiaoyang, LYU Jie, ZHANG Dengfeng. Principal component selection method for hyperspectral remote sensing images based on spatial statistics[J]. Remote Sensing for Natural Resources, 2022, 34(2): 37-46.
[15] QU Haicheng, WAND Yaxuan, SHEN Lei. Hyperspectral super-resolution combining multi-receptive field features with spectral-spatial attention[J]. Remote Sensing for Natural Resources, 2022, 34(1): 43-52.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech