Improved Transformer-based hyperspectral image classification method for surface features: A case study of the Yellow River Delta
LI Wei1(), FAN Yanguo1(), ZHOU Peixi2
1. College of Oceanography and Space Informatics,China University of Petroleum(East China),Qingdao 266580, China 2. Qingdao Hongyi Tiantu Information Technology Co., Ltd., Qingdao 266555, China
Hyperspectral technology has become the major means of coastal wetland monitoring. However, traditional hyperspectral classification methods usually face challenges such as insufficient feature extraction, the same surface features corresponding to different spectra, and fragmented scenes. To solve these problems, this study proposed a new classification method by applying Transformer to hyperspectral classification. This vision Transformer (ViT)-based method expanded the receptive field by learning global spatial features using non-local technology, thus overcoming the insufficient extraction of discriminant features. Meanwhile, this method enhanced the cross-layer information interchange through cross-layer adaptive residual connection, thus eliminating information loss. This study, taking NC16 and NC13 wetland datasets of the Yellow River Delta as experimental data, compared the classification method proposed in this study to support vector machine (SVM), one-dimensional convolution neural network (1DCNN), contextual deep convolution neural network (CDCNN), spectral-spatial residual network (SSRN), hybrid spectral network (HybridSN), and ViT. The comparison results show that the new method yielded significantly elevated overall accuracy (OA) of up to 96.24% and 73.84%, average accuracy (AA) reaching 83.42% and 74.87%, and Kappa coefficients of up to 94.80% and 68.94%, respectively for the two datasets.
李薇, 樊彦国, 周培希. 改进Transformer的高光谱图像地物分类方法——以黄河三角洲为例[J]. 自然资源遥感, 2024, 36(3): 137-145.
LI Wei, FAN Yanguo, ZHOU Peixi. Improved Transformer-based hyperspectral image classification method for surface features: A case study of the Yellow River Delta. Remote Sensing for Natural Resources, 2024, 36(3): 137-145.
Xia Y, Fang C, Lin H, et al. Spatiotemporal evolution of wetland eco-hydrological connectivity in the Poyang Lake area based on long time-series remote sensing images[J]. Remote Sensing, 2021, 13(23):4812.
[2]
López-Tapia S, Ruiz P, Smith M, et al. Machine learning with high-resolution aerial imagery and data fusion to improve and automate the detection of wetlands[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 105:102581.
[3]
Sun S, Wang Y, Song Z, et al. Modelling aboveground biomass carbon stock of the Bohai rim coastal wetlands by integrating remote sensing,terrain,and climate data[J]. Remote Sensing, 2021, 13(21):4321.
Wang X, Fan Y G. Hyperspectral image classification based on modified DenseNet and spatial spectrum attention mechanism[J]. Laser & Optoelectronics Progress, 2022, 59(2):0210014.
Zheng Z S, Liu H X, Wang Z H, et al. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2):105-111.doi:10.6046/zrzyyg.2022100.
[6]
Duan P, Kang X, Li S, et al. Multichannel pulse-coupled neural network-based hyperspectral image visualization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2444-2456.
Jia S P, Gao H J, Hang X. Research progress on image recognition technology of crop pests and diseases based on deep learning[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(s1):313-317.
[8]
Chen Y, Lin Z, Zhao X, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2094-2107.
[9]
Mou L, Ghamisi P, Zhu X X. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[10]
Paoletti M E, Haut J M, Fernandez-Beltran R, et al. Capsule networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(4):2145-2160.
[11]
Zhu L, Chen Y, Ghamisi P, et al. Generative adversarial networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9):5046-5063.
Chen Y H, Wang B, Yan Q Y, et al. Hyperspectral remote-sensing classification combining transformer and multiscale residual mechanisms[J]. Laser & Optoelectronics Progress, 2023, 60(12):3788/LOP220921.
[13]
Sun L, Zhao G, Zheng Y, et al. Spectral-spatial feature tokenization transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5522214.
[14]
Hong D, Han Z, Yao J, et al. SpectralFormer:Rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5518615.
[15]
Zhong Z, Li Y, Ma L, et al. Spectral-spatial transformer network for hyperspectral image classification:A factorized architecture search framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5514715.
[16]
Liu K, Sun W, Shao Y, et al. Mapping coastal wetlands using transformer in transformer deep network on China ZY1-02D hyperspectral satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:3891-3903.
[17]
Gao Y, Song X, Li W, et al. Fusion classification of HSI and MSI using a spatial-spectral vision transformer for wetland biodiversity estimation[J]. Remote Sensing, 2022, 14(4):850.
[18]
Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J/OL]. 2020.arXiv,2010.11929[2023-6-17]. https://arxiv.org/abs/2010.11929v1.
[19]
Wang C, Bai X, Zhou L, et al. Hyperspectral image classification based on non-local neural networks[C]// IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium.Yokohama,Japan.IEEE, 2019: 584-587.
[20]
Ma H, Liu G, Yuan Y. Enhanced non-local cascading network with attention mechanism for hyperspectral image denoising[C]// 2020 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Barcelona,Spain.IEEE, 2020:2448-2452.
[21]
Lei R, Zhang C, Du S, et al. A non-local capsule neural network for hyperspectral remote sensing image classification[J]. Remote Sensing Letters, 2021, 12(1):40-49.
[22]
He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks[C]// European Conference on Computer Vision.Cham:Springer, 2016:630-645.
[23]
Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,HI,USA.IEEE, 2017:2261-2269.
[24]
Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer, 2015:234-241.
[25]
He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).June 27-30,2016.Las Vegas,NV,USA.IEEE, 2016:770-778.
[26]
Hinton G E, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. Computer Science, 2015, 14(7):38-39.
[27]
Xie Z, Hu J, Kang X, et al. Multilayer global spectral-spatial attention network for wetland hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5518913.
[28]
Melgani F, Bruzzone L. Classification of hyperspectral remote sensing images with support vector machines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(8):1778-1790.
[29]
Hu W, Huang Y, Wei L, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015:258619.
[30]
Lee H, Kwon H. Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2017, 26(10):4843-4855.
[31]
Zhong Z, Li J, Luo Z, et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):847-858.
[32]
Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[33]
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017,30.
2024, Vol. 36 
), 樊彦国1(
