Please wait a minute...
 
Remote Sensing for Natural Resources    2022, Vol. 34 Issue (3) : 27-32     DOI: 10.6046/zrzyyg.2021271
|
Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information
ZHANG Pengqiang(), GAO Kuiliang(), LIU Bing, TAN Xiong
Information Engineering University, Zhengzhou 450001, China
Download: PDF(3088 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

The local convolution operation in convolutional neural networks cannot fully learn the global semantic information in hyperspectral images. Given this, this study designed a novel deep network model based on Transformer in order to further improve the classification precision of hyperspectral images. Firstly, this study reduced the dimensionality of hyperspectral images using the principal component analysis method and selected the neighborhood data around pixels as input samples to fully utilize the spatial-spectral information in the images. Secondly, the convolutional layer was used to transform the input samples into sequential characteristic vectors. Finally, image classification was conducted using the designed deep Transformer network. The multi-head attention mechanism in the Transformer model can make full use of the rich discriminative information. Experimental results show that the method proposed in this study can achieve better classification performance than the existing convolutional neural network model.
Keywords hyperspectral image classification      Transformer      deep learning      self-attention mechanism     
ZTFLH:  TP751  
Corresponding Authors: GAO Kuiliang     E-mail: zpq1978@163.com;gokling1219@163.com
Issue Date: 21 September 2022
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Pengqiang ZHANG
Kuiliang GAO
Bing LIU
Xiong TAN
Cite this article:   
Pengqiang ZHANG,Kuiliang GAO,Bing LIU, et al. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
URL:  
https://www.gtzyyg.com/EN/10.6046/zrzyyg.2021271     OR     https://www.gtzyyg.com/EN/Y2022/V34/I3/27
Fig.1  Basic structure of Transformer
Fig.2  Multi-head attention mechanism
Fig.3  Proposed network model
Fig.4  Classification of Salinas dataset
Fig.5  Classification of Indian Pines dataset
序号 类名称 SVM SSDCNN 3D-
CNN		 RES-3D
-CNN		 本文
方法
各类别分类精度/
%		 1 椰菜_绿_野草_1 99.20 98.90 98.75 100 91.74
2 椰菜_绿_野草_2 99.62 89.43 99.25 98.12 100.00
3 休耕地 99.70 100 99.14 99.70 100.00
4 粗糙的休耕地 99.50 99.93 99.50 99.86 100.00
5 平滑的休耕地 96.75 96.56 99.74 99.81 99.81
6 残株 99.42 99.97 99.62 100 99.87
7 芹菜 99.36 97.18 98.21 99.55 100.00
8 未结果实的葡萄 84.75 77.61 89.57 88.24 100.00
9 正在开发的葡萄园土壤 99.10 99.11 99.98 99.50 99.98
10 开始衰老的玉米 93.29 99.69 95.01 98.66 99.82
11 长叶莴苣(4周) 97.85 99.91 95.78 100 100.00
12 长叶莴苣(5周) 99.84 100 99.79 99.95 99.95
13 长叶莴苣(6周) 98.58 99.89 100.00 99.45 99.89
14 长叶莴苣(7周) 95.79 99.53 97.99 100 100.00
15 未结果实的葡萄园 65.74 96.68 82.12 91.98 96.40
16 葡萄园小路 98.89 99.34 99.43 99.45 100.00
OA/% 91.20 93.61 94.67 96.12 99.13
AA/% 95.46 97.11 97.12 98.39 99.22
Kappa 0.902 0 0.929 1 0.940 7 0.956 9 0.990 3
Tab.1  Classification results of Salinas dataset
序号 类名称 SVM SSDCNN 3D-
CNN		 RES-3D
-CNN		 本文
方法
各类别分类精度/
%		 1 免耕玉米 75.78 88.38 93.07 94.05 99.64
2 少耕玉米 71.12 94.34 96.27 99.04 97.42
3 草-牧场 89.10 97.52 98.55 98.96 100.00
4 草-树 96.15 99.86 97.12 99.59 97.73
5 堆积干草 99.79 100.00 100.00 100.00 100.00
6 免耕大豆 69.98 92.49 96.81 97.02 98.37
7 少耕大豆 89.22 87.17 91.85 91.28 99.59
8 大豆 79.79 99.33 97.47 99.33 97.85
9 树木 99.68 96.36 98.42 99.21 98.98
OA/% 84.57 92.81 95.41 96.12 98.96
AA/% 85.62 95.05 96.62 97.61 98.84
Kappa 0.820 9 0.915 9 0.946 3 0.954 7 0.987 8
Tab.2  Classification results of Indian Pines dataset
Fig.6  Influence of the number of training samples on classification accuracy
[1] He L, Li J, Liu C, et al. Recent advances on spectral-spatial hyperspectral image classification:An overview and new guidelines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(3):1579-1597.
doi: 10.1109/TGRS.2017.2765364 url: http://ieeexplore.ieee.org/document/8101519/
[2] Ghamisi P, Plaza J, Chen Y, et al. Advanced spectral classifiers for hyperspectral images:A review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(1):8-32.
[3] Tao C, Pan H, Li Y, et al. Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12):2438-2442.
doi: 10.1109/LGRS.2015.2482520 url: http://ieeexplore.ieee.org/document/7296592/
[4] Li T, Zhang J, Zhang Y. Classification of hyperspectral image based on deep belief networks[C]// Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), 2014.
[5] Zhang X R, Sun Y J, Jiang K, et al. Spatial sequential recurrent neural network for hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11):4141-4155.
doi: 10.1109/JSTARS.2018.2844873 url: https://ieeexplore.ieee.org/document/8399509/
[6] Xu Q, Xiao Y, Wang D, et al. CSA-MSO3DCNN:Multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification[J]. Remote Sensing, 2020, 12(1):188.
doi: 10.3390/rs12010188 url: https://www.mdpi.com/2072-4292/12/1/188
[7] Gao K, Guo W, Yu X, et al. Deep induction network for small samples classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3462-3477.
doi: 10.1109/JSTARS.2020.3002787 url: https://ieeexplore.ieee.org/document/9119099/
[8] 高奎亮, 张鹏强, 余旭初, 等. 基于Network In Network网络结构的高光谱影像分类方法[J]. 测绘科学技术学报, 2019, 36(5):500-504,510.
[8] Gao K L, Zhang P Q, Yu X C, et al. Classification method of hyperspectral image based on Network In Network structure[J]. Journal of Geomatics Science and Technology, 2019, 36(5):500-504,510.
[9] Li Y, Zhang H, Shen Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1):67.
doi: 10.3390/rs9010067 url: http://www.mdpi.com/2072-4292/9/1/67
[10] Xu X, Li J, Li S. Multiview intensity-based active learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):669-680.
doi: 10.1109/TGRS.2017.2752738 url: http://ieeexplore.ieee.org/document/8059873/
[11] He X, Chen Y. Transferring CNN ensemble for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5):876-880.
doi: 10.1109/LGRS.2020.2988494 url: https://ieeexplore.ieee.org/document/9084237/
[12] Mou L, Ghamisi P, Zhu X X. Unsupervised spectral-spatial feature learning via deep residual Conv-Deconv network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(1):391-406.
doi: 10.1109/TGRS.2017.2748160 url: http://ieeexplore.ieee.org/document/8082108/
[13] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Thirty-first Conference on Neural Information Processing Systems, 2017.
[14] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16[Z]. Transformers for Image Recognition at Scale, 2020.
[15] Yue J, Zhao W, Mao S, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[J]. Remote Sensing Letters, 2015, 6(4-6):468-477.
doi: 10.1080/2150704X.2015.1047045 url: http://www.tandfonline.com/doi/full/10.1080/2150704X.2015.1047045
[16] 刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1):53-63.
[16] Liu B, Yu X C, Zhang P Q, et al. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):53-63.
[1] NIU Xianghua, HUANG Wei, HUANG Rui, JIANG Sili. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.
[2] JIANG Zhuoran, ZHOU Xinxin, CAO Wei, WANG Yahua, WU Changbin. Intelligent detection of crab ponds using remote sensing images based on a cooperative interpretation mechanism[J]. Remote Sensing for Natural Resources, 2023, 35(3): 25-34.
[3] LIU Li, DONG Xianmin, LIU Juan. A performance evaluation method for semantic segmentation models of remote sensing images considering surface features[J]. Remote Sensing for Natural Resources, 2023, 35(3): 80-87.
[4] QIU Lei, ZHANG Xuezhi, HAO Dawei. VideoSAR moving target detection and tracking algorithm based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(2): 157-166.
[5] ZHANG Xian, LI Wei, CHEN Li, YANG Zhaoying, DOU Baocheng, LI Yu, CHEN Haomin. Research progress and prospect of remote sensing-based feature extraction of opencast mining areas[J]. Remote Sensing for Natural Resources, 2023, 35(2): 25-33.
[6] DIAO Mingguang, LIU Yong, GUO Ningbo, LI Wenji, JIANG Jikang, WANG Yunxiao. Mask R-CNN-based intelligent identification of sparse woods from remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 97-104.
[7] ZHANG Ke, ZHANG Gengsheng, WANG Ning, WEN Jing, LI Yu, YANG Jun. A forecasting method for water table depths in areas with power transmission lines based on remote sensing and deep learning models[J]. Remote Sensing for Natural Resources, 2023, 35(1): 213-221.
[8] HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[9] ZHAO Linghu, YUAN Xiping, GAN Shu, HU Lin, QIU Mingyu. An information extraction model of roads from high-resolution remote sensing images based on improved Deeplabv3+[J]. Remote Sensing for Natural Resources, 2023, 35(1): 107-114.
[10] LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(4): 22-32.
[11] TAN Hai, ZHANG Rongjun, FAN Wenfeng, ZHANG Yifang, XU Hang. Classification and detection of radiation anomalies in Chinese optical satellite images by integrating multi-scale features[J]. Remote Sensing for Natural Resources, 2022, 34(4): 97-104.
[12] SU Wei, LIN Yangyang, YUE Wen, CHEN Yingbiao. Identification of mariculture areas in Guangdong Province and remote sensing monitoring of their spatial and temporal changes based on the U-Net convolutional neural network[J]. Remote Sensing for Natural Resources, 2022, 34(4): 33-41.
[13] ERPAN Anwar, MAMAT Sawut, MAIHEMUTI Balati. Recognition of cotton distribution based on GF-2 images and Unet model[J]. Remote Sensing for Natural Resources, 2022, 34(2): 242-250.
[14] SUN Yu, HUANG Liang, ZHAO Junsan, CHANG Jun, CHEN Pengdi, CHENG Feifei. High spatial resolution automatic detection of bridges with high spatial resolution remote sensing images based on random erasure and YOLOv4[J]. Remote Sensing for Natural Resources, 2022, 34(2): 97-104.
[15] WANG Huajun, GE Xiaosan. Lightweight DeepLabv3+ building extraction method from remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(2): 128-135.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech