Please wait a minute...
 
Remote Sensing for Natural Resources    2025, Vol. 37 Issue (5) : 101-112     DOI: 10.6046/zrzyyg.2024191
|
Hyperspectral remote sensing image classification using improved residual 3D-CNN and neighborhood attention
PAN Zengying1,2(), WU Ruijiao3, LIN Yifeng1,2, WENG Qian1,2,4(), LIN Jiawen1,2,4
1. Fuzhou University College of Computer and Data Science，Fuzhou 350116，China
2. Fujian Key Laboratory of Network Computing and Intelligent Information Processing（Fuzhou University），Fuzhou 350116，China
3. Fujian Geologic Surveying and Mapping Institute，Fuzhou 350116，China
4. Key Lab of Spatial Data Mining & Information Sharing，Ministry of Education（Fuzhou University），Fuzhou 350116，China
Download: PDF(6458 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Hyperspectral remote sensing image classification has attracted widespread attention，yet the performance of classification methods remains greatly limited by challenges such as spectral variability （same object with different spectra），spectral confusion （different objects with similar spectra），and limited availability of training samples. To fully exploit the spatial-spectral features of hyperspectral images，this study proposed an improved network integrating residual convolution and neighborhood attention mechanisms. The proposed method consists of：（1） a residual-based spectral feature extraction module combining residual connections and a 3D convolutional neural network （3D-CNN）；（2） a spatial-spectral feature fusion module using mixed convolutions；and （3） a neighborhood attention module designed to enhance the model's ability to focus on homogeneous regions. Experiments were conducted on three public hyperspectral datasets-Indian Pines，Pavia University，and Houston 2013. The results demonstrate that the proposed method achieves higher classification accuracy compared to recent state-of-the-art approaches. Using less than 10% of the samples for training，it attains overall accuracies of 99.39%，99.67%，and 98.64%，respectively，confirming its capability for high-accuracy classification under small-sample conditions.
Keywords hyperspectral image classification      convolutional neural network      residual connection      neighborhood attention     
ZTFLH:  TP79  
Issue Date: 28 October 2025
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Zengying PAN
Ruijiao WU
Yifeng LIN
Qian WENG
Jiawen LIN
Cite this article:   
Zengying PAN,Ruijiao WU,Yifeng LIN, et al. Hyperspectral remote sensing image classification using improved residual 3D-CNN and neighborhood attention[J]. Remote Sensing for Natural Resources, 2025, 37(5): 101-112.
URL:  
https://www.gtzyyg.com/EN/10.6046/zrzyyg.2024191     OR     https://www.gtzyyg.com/EN/Y2025/V37/I5/101
Fig.1  Improved residual 3D-CNN and nearest neighbor attention network
Fig.2  Residual based spectral feature extraction module
Fig.3  Space spectral feature fusion module
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
苜蓿 22.93 55.61 72.93 79.76 79.27 89.51 88.29 97.56
未翻耕的玉米地 59.95 78.06 84.92 95.35 98.67 97.11 98.79 98.96
翻耕过的玉米地 51.02 82.53 89.21 93.36 98.33 96.49 95.92 98.89
玉米地 38.78 82.35 91.60 88.69 97.56 95.82 97.32 98.50
牧草区 73.72 95.52 96.74 92.34 98.92 96.46 98.32 98.69
草地与树木 86.51 97.84 99.47 97.69 99.27 99.74 99.89 99.82
已收割的牧草区 45.60 42.80 66.40 64.80 92.40 99.20 100.00 99.60
风干的草料 94.26 99.49 98.09 97.19 99.81 99.79 99.88 99.93
燕麦 18.89 65.56 9.44 94.44 99.44 84.44 97.78 98.33
未翻耕的大豆田 48.00 76.99 86.82 96.49 98.89 98.74 99.22 99.05
翻耕过的大豆田 67.89 84.51 97.24 95.71 99.38 98.45 99.50 99.67
已清理的大豆田 43.73 82.73 98.45 91.46 99.29 95.13 98.65 99.16
小麦 93.35 99.19 99.68 98.11 100.00 99.73 99.78 100.00
树林 92.99 98.13 92.68 99.08 99.93 99.87 99.99 100.00
建筑物、草地、树木、车道 39.39 89.57 89.34 88.1 98.67 98.30 98.79 100.00
石、钢、塔、楼 89.76 98.81 98.57 81.79 98.10 94.76 94.88 96.79
OA/% 66.67 86.77 92.84 95.08 99.03 98.04 98.92 99.39
AA/% 60.42 83.11 85.72 90.9 97.37 96.47 97.94 99.06
Kappa 0.619 8 0.847 5 0.918 8 0.943 9 0.988 9 0.977 6 0.987 7 0.993 0
Tab.1  Classification Results of Different Methods on IP Datasets
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
沥青路面 87.81 95.72 99.96 96.58 99.87 98.75 99.57 99.99
草地 90.85 97.50 99.60 99.77 99.97 99.91 99.88 99.98
碎石 57.84 69.74 92.47 86.49 94.52 89.45 89.90 97.49
树木 85.84 98.36 98.66 88.23 99.07 96.19 97.81 98.61
涂漆金属板 98.75 99.94 100.00 86.66 99.98 100.00 100.00 99.98
裸土 72.67 84.88 99.69 99.60 100.00 99.95 99.97 99.94
沥青 79.44 89.11 99.93 90.75 99.46 99.86 99.51 100.00
自阻砖 84.66 90.74 98.53 89.17 98.46 93.43 96.43 99.28
阴影 99.52 99.67 99.91 78.88 99.94 93.72 97.60 98.84
OA/% 85.82 93.72 99.18 95.71 99.48 98.26 98.85 99.67
AA/% 84.15 91.74 98.75 90.68 99.03 96.81 97.85 99.35
Kappa 0.812 3 0.916 1 0.989 2 0.943 0 0.993 1 0.976 9 0.984 8 0.995 6
Tab.2  Classification results of different methods on the PU dataset
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
健康的草地 93.90 94.65 93.36 91.83 96.37 94.77 94.85 96.83
受压力的草地 94.81 97.63 96.54 95.50 99.13 97.95 98.85 99.21
人造草皮 95.39 95.44 92.63 97.98 99.24 99.31 99.63 99.89
树木 94.03 98.60 99.87 95.53 99.93 97.52 97.66 99.93
土壤 95.86 97.37 99.61 100.00 100.00 100.00 100.00 100.00
水 68.01 67.21 78.16 85.37 84.04 83.75 96.25 95.44
住宅区 80.29 95.12 93.66 94.52 98.24 98.68 98.94 99.05
商业区 78.10 84.39 91.09 91.14 94.93 94.52 94.16 94.94
道路 73.29 89.06 83.47 93.73 98.55 97.80 97.79 98.11
高速公路 71.85 87.05 88.26 98.77 99.35 99.47 99.86 99.97
铁路 63.66 92.69 90.84 99.85 99.78 98.32 99.95 100.00
停车场1 75.47 92.18 88.17 96.51 97.84 96.82 96.77 99.12
停车场2 23.19 88.01 80.87 89.78 91.36 89.21 92.62 92.29
网球场 89.44 98.91 94.84 73.33 99.96 99.89 100.00 100.00
跑道 94.79 100.00 99.34 97.99 100.00 99.89 100.00 100.00
OA/% 81.11 92.91 92.14 95.19 98.21 97.45 97.96 98.64
AA/% 79.47 91.89 91.38 93.46 97.25 96.53 97.82 98.32
Kappa 0.794 7 0.923 0 0.914 5 0.947 7 0.980 6 0.972 3 0.977 8 0.985 2
Tab.3  Classification results of different methods on the HO dataset
Fig.4  Comparison of classification maps of various methods on the IP dataset
Fig.5  Comparison of classification maps of various methods on the PU dataset
Fig.6  Comparison of classification maps of various methods on the HO dataset
Fig.7  Visualization of t-SNE features for 3 datasets
Fig.8  Accuracy comparison of different training samples under 3 data sets
数据集 精度 Baseline RSFEM NAM RSFEM+NAM
IP OA/% 99.04 99.38 99.11 99.39
AA/% 98.31 99.06 98.38 99.06
Kappa 0.989 1 0.993 0 0.989 9 0.993 0
PU OA/% 99.26 99.45 99.30 99.67
AA/% 98.19 98.63 98.34 99.35
Kappa 0.990 2 0.992 7 0.990 7 0.995 6
HO OA/% 98.39 98.53 98.50 98.64
AA/% 98.17 98.13 98.24 98.32
Kappa 0.982 5 0.984 0 0.983 7 0.985 2
Tab.4  Ablation comparison of each module
Fig.9  Accuracy line plots of different PCA dimensionality reduction dimensions in 3 datasets
数据集 精度 7 9 11 13 15
IP OA/% 99.27 99.39 99.22 98.97 98.65
AA/% 98.60 99.06 98.85 95.83 91.74
Kappa 0.991 7 0.993 0 0.991 1 0.988 3 0.984 6
PU OA/% 99.19 99.42 99.58 99.67 99.64
AA/% 98.56 99.04 99.18 99.35 99.33
Kappa 0.989 3 0.992 3 0.994 4 0.995 6 0.995 2
HO OA/% 98.42 98.64 98.52 98.42 98.34
AA/% 97.98 98.32 97.95 97.78 97.68
Kappa 0.982 8 0.985 2 0.983 9 0.982 8 0.981 9
Tab.5  Comparison of classification accuracy for different patch sizes
[1] 张涛, 王彬沣, 付莹, 等. 基于深度学习的光谱图像超分辨率综述[J]. 中国图象图形学报, 2024, 29(8):2113-2136.
[1] Zhang T, Wang B F, Fu Y, et al. Deep learning-based spectral image super-resolution:A survey[J]. Journal of Image and Graphics, 2024, 29(8):2113-2136.
[2] 马晓瑞, 哈林, 谌敦斌, 等. 融合特征优化的跨数据集高光谱图像分类[J]. 中国图象图形学报, 2024, 29(8):2175-2187.
[2] Ma X R, Ha L, Shen D B, et al. Cross-dataset hyperspectral image classification based on fusion feature optimization[J]. Journal of Image and Graphics, 2024, 29(8):2175-2187.
[3] 袁静文, 武辰, 杜博, 等. 高分五号高光谱遥感影像的城市土地利用景观格局分析[J]. 遥感学报, 2020, 24(4):465-478.
[3] Yuan J W, Wu C, Du B, et al. Analysis of landscape pattern on urban land use based on GF-5 hyperspectral data[J]. Journal of Remote Sensing, 2020, 24(4):465-478.
[4] 王书伟, 席磊, 邱霜, 等. 高光谱遥感森林资源监测原理与应用[J]. 安徽农业科学, 2023, 51(15):111-114,197.
[4] Wang S W, Xi L, Qiu S, et al. Principle and application of hyperspectral remote sensing forest resource monitoring[J]. Journal of Anhui Agricultural Sciences, 2023, 51(15):111-114,197.
[5] 杨燕杰. 基于高光谱的高海拔地区矿物信息提取与分析[J]. 世界核地质科学, 2023, 40(4):1020-1028.
[5] Yang Y J. The alteration mineral information extraction and analysis of hyperspectral data in high-altitude area[J]. World Nuclear Geoscience, 2023, 40(4):1020-1028.
[6] 吕欢欢, 张峻通, 张辉. 多尺度特征与双注意力机制的高光谱影像分类[J]. 光电子·激光, 2024, 35(2):143-154.
[6] Lyu H H, Zhang J T, Zhang H. Multi-scale feature and dual-attention mechanism for hyperspectral image classification[J]. Journal of Optoelectronics·Laser, 2024, 35(2):143-154.
[7] Chen Y, Lin Z, Zhao X, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2094-2107.
[8] Liu Q, Zhou F, Hang R, et al. Bidirectional-convolutional LSTM based spectral-spatial feature learning for hyperspectral image classification[J]. Remote Sensing, 2017, 9(12):1330.
[9] Mou L, Ghamisi P, Zhu X X. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[10] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016:770-778.
[11] Ren S, He K, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[12] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE,2017:640-651.
[13] Hu W, Huang Y, Wei L, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015(1):258619.
[14] Makantasis K, Karantzalos K, Doulamis A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, 2015:4959-4962.
[15] Zhu M, Jiao L, Liu F, et al. Residual spectral-spatial attention network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(1):449-462.
[16] Kang X, Zhuo B, Duan P. Dual-path network-based hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(3):447-451.
doi: 10.1109/LGRS.2018.2873476
[17] Hamida A B, Benoit A, Lambert P, et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8):4420-4434.
[18] Zhong Z, Li J, Luo Z, et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):847-858.
[19] 郑宗生, 王政翰, 王振华, 等. 改进3D-Octave卷积的高光谱图像分类方法[J]. 自然资源遥感, 2024, 36(4):82-91.doi:10.6046/zrzyyg.2023171.
[19] Zheng Z S, Wang Z H, Wang Z H, et al. Improved 3D-Octave convolutional hyperspectral image classification method[J]. Remote Sensing for Natural Resources, 2024, 36(4):82-91.doi:10.6046/zrzyyg.2023171.
[20] Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[21] Roy S K, Manna S, Song T, et al. Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9):7831-7843.
[22] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J/OL]. arXiv, 2020. https://arxiv.org/abs/2010.11929.
url: https://arxiv.org/abs/2010.11929
[23] Roy S K, Deria A, Shah C, et al. Spectral-spatial morphological attention Transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61:5503615.
[24] Chen N, Fang L, Xia Y, et al. Spectral query spatial:Revisiting the role of center pixel in transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62:5402714.
[25] Ma C, Wan M, Wu J, et al. Light self-Gaussian-attention vision Transformer for hyperspectral image classification[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72:5015712.
[26] D’Ascoli S, Touvron H, et al. ConViT:Improving vision Transfor-mers with soft convolutional inductive biases[C]//International conference on machine learning. PMLR,2021:2286-2296.
[27] Gao H, Chen Z, Xu F. Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 107:102687.
[28] Zhao Z, Hu D, Wang H, et al. Convolutional Transformer network for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:6009005.
[29] Debes C, Merentitis A, Heremans R, et al. Hyperspectral and LiDAR data fusion:Outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2405-2418.
[30] Zhang L, Zhang L, Du B. Deep learning for remote sensing data:A technical tutorial on the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2016, 4(2):22-40.
[1] ZHENG Zongsheng, GAO Meng, ZHOU Wenhuan, WANG Zhenghan, HUO Zhijun, ZHANG Yuewei. Densely connected multiscale semantic segmentation for land cover based on the iterative optimization strategy for samples[J]. Remote Sensing for Natural Resources, 2025, 37(2): 11-18.
[2] HUANG Chuan, LI Yaqin, QI Yueran, WEI Xiaoyan, SHAO Yuanzheng. A hyperspectral unmixing and few-shot classification method based on 3DCAE network[J]. Remote Sensing for Natural Resources, 2025, 37(1): 8-14.
[3] LI Lei, SUN Xiyan, JI Yuanfa, FU Wentao. Hyperspectral image classification based on superpixel segmentation and extended multi-attribute profiles[J]. Remote Sensing for Natural Resources, 2023, 35(4): 114-121.
[4] DENG Dingzhu. Deep learning-based cloud detection method for multi-source satellite remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(4): 9-16.
[5] HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[6] SUN Sheng, MENG Zhimin, HU Zhongwen, YU Xu. Application of multi-scale and lightweight CNN in SAR image-based surface feature classification[J]. Remote Sensing for Natural Resources, 2023, 35(1): 27-34.
[7] ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
[8] LIAO Kuo, NIE Lei, YANG Zeyu, ZHANG Hongyan, WANG Yanjie, PENG Jida, DANG Haofei, LENG Wei. Classification of tea garden based on multi-source high-resolution satellite images using multi-dimensional convolutional neural network[J]. Remote Sensing for Natural Resources, 2022, 34(2): 152-161.
[9] YANG Zhaoying, HAN Lingyi, ZHENG Xiangxiang, LI Wenji, FENG Lei, WANG Yi, YANG Yongpeng. Landslide identification using remote sensing images and DEM based on convolutional neural network: A case study of loess landslide[J]. Remote Sensing for Natural Resources, 2022, 34(2): 224-230.
[10] KONG Ailing, ZHANG Chengming, LI Feng, HAN Yingjuan, SUN Huanying, DU Manfei. Knowledge-based remote sensing image fusion method[J]. Remote Sensing for Natural Resources, 2022, 34(2): 47-55.
[11] YU Xinli, SONG Yan, YANG Miao, HUANG Lei, ZHANG Yanjie. Multi-model and multi-scale scene recognition of shipbuilding enterprises based on convolutional neural network with spatial constraints[J]. Remote Sensing for Natural Resources, 2021, 33(4): 72-81.
[12] HAN Yanling, CUI Pengxia, YANG Shuhu, LIU Yekun, WANG Jing, ZHANG Yun. Classification of hyperspectral image based on feature fusion of residual network[J]. Remote Sensing for Land & Resources, 2021, 33(2): 11-19.
[13] QIU Yifan, CHAI Dengfeng. A deep learning method for Landsat image cloud detection without manually labeled data[J]. Remote Sensing for Land & Resources, 2021, 33(1): 102-107.
[14] LIU Zhao, ZHAO Tong, LIAO Feifan, LI Shuai, LI Haiyang. Research and comparative analysis on urban built-up area extraction methods from high-resolution remote sensing image based on semantic segmentation network[J]. Remote Sensing for Land & Resources, 2021, 33(1): 45-53.
[15] WEI Hongyu, ZHAO Yindi, DONG Jihong. Cooling tower detection based on the improved RetinaNet[J]. Remote Sensing for Land & Resources, 2020, 32(4): 68-73.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech