Please wait a minute...
 
自然资源遥感  2025, Vol. 37 Issue (5): 101-112    DOI: 10.6046/zrzyyg.2024191
  技术方法 本期目录 | 过刊浏览 | 高级检索 |
改进的残差式3D-CNN和近邻注意力的高光谱遥感图像分类
潘增滢1,2(), 吴瑞姣3, 林易丰1,2, 翁谦1,2,4(), 林嘉雯1,2,4
1.福州大学计算机与大数据学院，福州 350116
2.福州大学福建省网络计算与智能信息处理重点实验室，福州 350116
3.福建省地质测绘院，福州 350116
4.福州大学空间数据挖掘与信息共享教育部重点实验室，福州 350116
Hyperspectral remote sensing image classification using improved residual 3D-CNN and neighborhood attention
PAN Zengying1,2(), WU Ruijiao3, LIN Yifeng1,2, WENG Qian1,2,4(), LIN Jiawen1,2,4
1. Fuzhou University College of Computer and Data Science，Fuzhou 350116，China
2. Fujian Key Laboratory of Network Computing and Intelligent Information Processing（Fuzhou University），Fuzhou 350116，China
3. Fujian Geologic Surveying and Mapping Institute，Fuzhou 350116，China
4. Key Lab of Spatial Data Mining & Information Sharing，Ministry of Education（Fuzhou University），Fuzhou 350116，China
全文: PDF(6458 KB)   HTML  
输出: BibTeX | EndNote (RIS)      
摘要 

在引起广泛关注的高光谱遥感图像分类中，同物异谱、同谱异物和少样本都大大限制了分类方法的性能。为了充分挖掘高光谱图像的空间-光谱特征，该文提出了一种改进的残差卷积和近邻注意力网络用于高光谱遥感图像分类。该方法包含3个部分：结合了残差连接和3D卷积神经网络（3D convolutional neural network，3D-CNN）的残差式光谱特征提取模块、使用混合卷积的空间-光谱特征融合模块、用于增强模型对同质区域的关注能力的近邻注意力模块。在3个公开的高光谱数据集Indian pines，Pavia University，Houston2013上的实验结果显示，相比近期先进高光谱分类方法，所提方法有更高的分类精度，且在使用10%以下训练样本的前提下总体精度可分别达到99.39%，99.67%和98.64%，实现了少样本下的高精度分类。
服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
潘增滢
吴瑞姣
林易丰
翁谦
林嘉雯
关键词 ： 高光谱图像分类,  卷积神经网络,  残差连接,  近邻注意力    
Abstract：

Hyperspectral remote sensing image classification has attracted widespread attention，yet the performance of classification methods remains greatly limited by challenges such as spectral variability （same object with different spectra），spectral confusion （different objects with similar spectra），and limited availability of training samples. To fully exploit the spatial-spectral features of hyperspectral images，this study proposed an improved network integrating residual convolution and neighborhood attention mechanisms. The proposed method consists of：（1） a residual-based spectral feature extraction module combining residual connections and a 3D convolutional neural network （3D-CNN）；（2） a spatial-spectral feature fusion module using mixed convolutions；and （3） a neighborhood attention module designed to enhance the model's ability to focus on homogeneous regions. Experiments were conducted on three public hyperspectral datasets-Indian Pines，Pavia University，and Houston 2013. The results demonstrate that the proposed method achieves higher classification accuracy compared to recent state-of-the-art approaches. Using less than 10% of the samples for training，it attains overall accuracies of 99.39%，99.67%，and 98.64%，respectively，confirming its capability for high-accuracy classification under small-sample conditions.
Key words： hyperspectral image classification    convolutional neural network    residual connection    neighborhood attention
收稿日期: 2024-05-24      出版日期: 2025-10-28
ZTFLH:  TP79  
基金资助:福建省自然科学基金项目“人机协同的自然资源要素提取关键技术研究”(2023J01432);国家自然科学基金项目“基于深度迁移学习网络的高分影像土地利用分类方法研究”(41801324)
通讯作者: 翁 谦（1983-），男，博士，副教授，主要从事计算机视觉和时空大数据研究。Email：fzuwq@fzu.edu.cn。
作者简介: 潘增滢（2000-），男，硕士研究生，主要从事高光谱遥感信息处理研究。Email：1157928777@qq.com。
引用本文:   
潘增滢, 吴瑞姣, 林易丰, 翁谦, 林嘉雯. 改进的残差式3D-CNN和近邻注意力的高光谱遥感图像分类[J]. 自然资源遥感, 2025, 37(5): 101-112.
PAN Zengying, WU Ruijiao, LIN Yifeng, WENG Qian, LIN Jiawen. Hyperspectral remote sensing image classification using improved residual 3D-CNN and neighborhood attention. Remote Sensing for Natural Resources, 2025, 37(5): 101-112.
链接本文:  
https://www.gtzyyg.com/CN/10.6046/zrzyyg.2024191      或      https://www.gtzyyg.com/CN/Y2025/V37/I5/101
Fig.1  改进的残差3D-CNN和近邻注意力网络
Fig.2  RSFEM模块
Fig.3  SSFFM模块
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
苜蓿 22.93 55.61 72.93 79.76 79.27 89.51 88.29 97.56
未翻耕的玉米地 59.95 78.06 84.92 95.35 98.67 97.11 98.79 98.96
翻耕过的玉米地 51.02 82.53 89.21 93.36 98.33 96.49 95.92 98.89
玉米地 38.78 82.35 91.60 88.69 97.56 95.82 97.32 98.50
牧草区 73.72 95.52 96.74 92.34 98.92 96.46 98.32 98.69
草地与树木 86.51 97.84 99.47 97.69 99.27 99.74 99.89 99.82
已收割的牧草区 45.60 42.80 66.40 64.80 92.40 99.20 100.00 99.60
风干的草料 94.26 99.49 98.09 97.19 99.81 99.79 99.88 99.93
燕麦 18.89 65.56 9.44 94.44 99.44 84.44 97.78 98.33
未翻耕的大豆田 48.00 76.99 86.82 96.49 98.89 98.74 99.22 99.05
翻耕过的大豆田 67.89 84.51 97.24 95.71 99.38 98.45 99.50 99.67
已清理的大豆田 43.73 82.73 98.45 91.46 99.29 95.13 98.65 99.16
小麦 93.35 99.19 99.68 98.11 100.00 99.73 99.78 100.00
树林 92.99 98.13 92.68 99.08 99.93 99.87 99.99 100.00
建筑物、草地、树木、车道 39.39 89.57 89.34 88.1 98.67 98.30 98.79 100.00
石、钢、塔、楼 89.76 98.81 98.57 81.79 98.10 94.76 94.88 96.79
OA/% 66.67 86.77 92.84 95.08 99.03 98.04 98.92 99.39
AA/% 60.42 83.11 85.72 90.9 97.37 96.47 97.94 99.06
Kappa 0.619 8 0.847 5 0.918 8 0.943 9 0.988 9 0.977 6 0.987 7 0.993 0
Tab.1  不同方法在IP数据集上的分类结果
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
沥青路面 87.81 95.72 99.96 96.58 99.87 98.75 99.57 99.99
草地 90.85 97.50 99.60 99.77 99.97 99.91 99.88 99.98
碎石 57.84 69.74 92.47 86.49 94.52 89.45 89.90 97.49
树木 85.84 98.36 98.66 88.23 99.07 96.19 97.81 98.61
涂漆金属板 98.75 99.94 100.00 86.66 99.98 100.00 100.00 99.98
裸土 72.67 84.88 99.69 99.60 100.00 99.95 99.97 99.94
沥青 79.44 89.11 99.93 90.75 99.46 99.86 99.51 100.00
自阻砖 84.66 90.74 98.53 89.17 98.46 93.43 96.43 99.28
阴影 99.52 99.67 99.91 78.88 99.94 93.72 97.60 98.84
OA/% 85.82 93.72 99.18 95.71 99.48 98.26 98.85 99.67
AA/% 84.15 91.74 98.75 90.68 99.03 96.81 97.85 99.35
Kappa 0.812 3 0.916 1 0.989 2 0.943 0 0.993 1 0.976 9 0.984 8 0.995 6
Tab.2  不同方法在PU数据集上的分类结果
类别 1D-CNN 2D-CNN SSRN HybridSN A2S2K morphFormer LSGA IR3NAN
健康的草地 93.90 94.65 93.36 91.83 96.37 94.77 94.85 96.83
受压力的草地 94.81 97.63 96.54 95.50 99.13 97.95 98.85 99.21
人造草皮 95.39 95.44 92.63 97.98 99.24 99.31 99.63 99.89
树木 94.03 98.60 99.87 95.53 99.93 97.52 97.66 99.93
土壤 95.86 97.37 99.61 100.00 100.00 100.00 100.00 100.00
水 68.01 67.21 78.16 85.37 84.04 83.75 96.25 95.44
住宅区 80.29 95.12 93.66 94.52 98.24 98.68 98.94 99.05
商业区 78.10 84.39 91.09 91.14 94.93 94.52 94.16 94.94
道路 73.29 89.06 83.47 93.73 98.55 97.80 97.79 98.11
高速公路 71.85 87.05 88.26 98.77 99.35 99.47 99.86 99.97
铁路 63.66 92.69 90.84 99.85 99.78 98.32 99.95 100.00
停车场1 75.47 92.18 88.17 96.51 97.84 96.82 96.77 99.12
停车场2 23.19 88.01 80.87 89.78 91.36 89.21 92.62 92.29
网球场 89.44 98.91 94.84 73.33 99.96 99.89 100.00 100.00
跑道 94.79 100.00 99.34 97.99 100.00 99.89 100.00 100.00
OA/% 81.11 92.91 92.14 95.19 98.21 97.45 97.96 98.64
AA/% 79.47 91.89 91.38 93.46 97.25 96.53 97.82 98.32
Kappa 0.794 7 0.923 0 0.914 5 0.947 7 0.980 6 0.972 3 0.977 8 0.985 2
Tab.3  不同方法在HO集上的分类结果
Fig.4  各个方法在IP数据集分类图对比
Fig.5  各个方法在PU数据集分类图对比
Fig.6  各个方法在HO数据集分类图对比
Fig.7  3个数据集的t-SNE特征可视化
Fig.8  3个数据集下不同训练样本精度对比
数据集 精度 Baseline RSFEM NAM RSFEM+NAM
IP OA/% 99.04 99.38 99.11 99.39
AA/% 98.31 99.06 98.38 99.06
Kappa 0.989 1 0.993 0 0.989 9 0.993 0
PU OA/% 99.26 99.45 99.30 99.67
AA/% 98.19 98.63 98.34 99.35
Kappa 0.990 2 0.992 7 0.990 7 0.995 6
HO OA/% 98.39 98.53 98.50 98.64
AA/% 98.17 98.13 98.24 98.32
Kappa 0.982 5 0.984 0 0.983 7 0.985 2
Tab.4  各个模块的消融对比
Fig.9  3个数据集下不同PCA降维后特征数对分类精度影响图
数据集 精度 7 9 11 13 15
IP OA/% 99.27 99.39 99.22 98.97 98.65
AA/% 98.60 99.06 98.85 95.83 91.74
Kappa 0.991 7 0.993 0 0.991 1 0.988 3 0.984 6
PU OA/% 99.19 99.42 99.58 99.67 99.64
AA/% 98.56 99.04 99.18 99.35 99.33
Kappa 0.989 3 0.992 3 0.994 4 0.995 6 0.995 2
HO OA/% 98.42 98.64 98.52 98.42 98.34
AA/% 97.98 98.32 97.95 97.78 97.68
Kappa 0.982 8 0.985 2 0.983 9 0.982 8 0.981 9
Tab.5  不同大小空间块分类精度对比
[1] 张涛, 王彬沣, 付莹, 等. 基于深度学习的光谱图像超分辨率综述[J]. 中国图象图形学报, 2024, 29(8):2113-2136.
Zhang T, Wang B F, Fu Y, et al. Deep learning-based spectral image super-resolution:A survey[J]. Journal of Image and Graphics, 2024, 29(8):2113-2136.
[2] 马晓瑞, 哈林, 谌敦斌, 等. 融合特征优化的跨数据集高光谱图像分类[J]. 中国图象图形学报, 2024, 29(8):2175-2187.
Ma X R, Ha L, Shen D B, et al. Cross-dataset hyperspectral image classification based on fusion feature optimization[J]. Journal of Image and Graphics, 2024, 29(8):2175-2187.
[3] 袁静文, 武辰, 杜博, 等. 高分五号高光谱遥感影像的城市土地利用景观格局分析[J]. 遥感学报, 2020, 24(4):465-478.
Yuan J W, Wu C, Du B, et al. Analysis of landscape pattern on urban land use based on GF-5 hyperspectral data[J]. Journal of Remote Sensing, 2020, 24(4):465-478.
[4] 王书伟, 席磊, 邱霜, 等. 高光谱遥感森林资源监测原理与应用[J]. 安徽农业科学, 2023, 51(15):111-114,197.
Wang S W, Xi L, Qiu S, et al. Principle and application of hyperspectral remote sensing forest resource monitoring[J]. Journal of Anhui Agricultural Sciences, 2023, 51(15):111-114,197.
[5] 杨燕杰. 基于高光谱的高海拔地区矿物信息提取与分析[J]. 世界核地质科学, 2023, 40(4):1020-1028.
Yang Y J. The alteration mineral information extraction and analysis of hyperspectral data in high-altitude area[J]. World Nuclear Geoscience, 2023, 40(4):1020-1028.
[6] 吕欢欢, 张峻通, 张辉. 多尺度特征与双注意力机制的高光谱影像分类[J]. 光电子·激光, 2024, 35(2):143-154.
Lyu H H, Zhang J T, Zhang H. Multi-scale feature and dual-attention mechanism for hyperspectral image classification[J]. Journal of Optoelectronics·Laser, 2024, 35(2):143-154.
[7] Chen Y, Lin Z, Zhao X, et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2094-2107.
[8] Liu Q, Zhou F, Hang R, et al. Bidirectional-convolutional LSTM based spectral-spatial feature learning for hyperspectral image classification[J]. Remote Sensing, 2017, 9(12):1330.
[9] Mou L, Ghamisi P, Zhu X X. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[10] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016:770-778.
[11] Ren S, He K, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
[12] Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE,2017:640-651.
[13] Hu W, Huang Y, Wei L, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015(1):258619.
[14] Makantasis K, Karantzalos K, Doulamis A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, 2015:4959-4962.
[15] Zhu M, Jiao L, Liu F, et al. Residual spectral-spatial attention network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(1):449-462.
[16] Kang X, Zhuo B, Duan P. Dual-path network-based hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(3):447-451.
doi: 10.1109/LGRS.2018.2873476
[17] Hamida A B, Benoit A, Lambert P, et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8):4420-4434.
[18] Zhong Z, Li J, Luo Z, et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):847-858.
[19] 郑宗生, 王政翰, 王振华, 等. 改进3D-Octave卷积的高光谱图像分类方法[J]. 自然资源遥感, 2024, 36(4):82-91.doi:10.6046/zrzyyg.2023171.
Zheng Z S, Wang Z H, Wang Z H, et al. Improved 3D-Octave convolutional hyperspectral image classification method[J]. Remote Sensing for Natural Resources, 2024, 36(4):82-91.doi:10.6046/zrzyyg.2023171.
[20] Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[21] Roy S K, Manna S, Song T, et al. Attention-based adaptive spectral-spatial kernel ResNet for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9):7831-7843.
[22] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J/OL]. arXiv, 2020. https://arxiv.org/abs/2010.11929.
[23] Roy S K, Deria A, Shah C, et al. Spectral-spatial morphological attention Transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61:5503615.
[24] Chen N, Fang L, Xia Y, et al. Spectral query spatial:Revisiting the role of center pixel in transformer for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62:5402714.
[25] Ma C, Wan M, Wu J, et al. Light self-Gaussian-attention vision Transformer for hyperspectral image classification[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72:5015712.
[26] D’Ascoli S, Touvron H, et al. ConViT:Improving vision Transfor-mers with soft convolutional inductive biases[C]//International conference on machine learning. PMLR,2021:2286-2296.
[27] Gao H, Chen Z, Xu F. Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 107:102687.
[28] Zhao Z, Hu D, Wang H, et al. Convolutional Transformer network for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:6009005.
[29] Debes C, Merentitis A, Heremans R, et al. Hyperspectral and LiDAR data fusion:Outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2405-2418.
[30] Zhang L, Zhang L, Du B. Deep learning for remote sensing data:A technical tutorial on the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2016, 4(2):22-40.
[1] 郑宗生, 高萌, 周文睆, 王政翰, 霍志俊, 张月维. 基于样本迭代优化策略的密集连接多尺度土地覆盖语义分割[J]. 自然资源遥感, 2025, 37(2): 11-18.
[2] 黄川, 李雅琴, 祁越然, 魏晓燕, 邵远征. 基于3D-CAE的高光谱解混及小样本分类方法[J]. 自然资源遥感, 2025, 37(1): 8-14.
[3] 李雷, 孙希延, 纪元法, 付文涛. 融合超像素和多属性形态学轮廓方法的高光谱图像分类[J]. 自然资源遥感, 2023, 35(4): 114-121.
[4] 邓丁柱. 基于深度学习的多源卫星遥感影像云检测方法[J]. 自然资源遥感, 2023, 35(4): 9-16.
[5] 郑宗生, 刘海霞, 王振华, 卢鹏, 沈绪坤, 唐鹏飞. 改进3D-CNN的高光谱图像地物分类方法[J]. 自然资源遥感, 2023, 35(2): 105-111.
[6] 胡建文, 汪泽平, 胡佩. 基于深度学习的空谱遥感图像融合综述[J]. 自然资源遥感, 2023, 35(1): 1-14.
[7] 孙盛, 蒙芝敏, 胡忠文, 余旭. 多尺度轻量化CNN在SAR图像地物分类中的应用[J]. 自然资源遥感, 2023, 35(1): 27-34.
[8] 廖廓, 聂磊, 杨泽宇, 张红艳, 王艳杰, 彭继达, 党皓飞, 冷伟. 基于多维卷积神经网络的多源高分辨率卫星影像茶园分类[J]. 自然资源遥感, 2022, 34(2): 152-161.
[9] 杨昭颖, 韩灵怡, 郑向向, 李文吉, 冯磊, 王轶, 杨永鹏. 基于卷积神经网络的遥感影像及DEM滑坡识别——以黄土滑坡为例[J]. 自然资源遥感, 2022, 34(2): 224-230.
[10] 孔爱玲, 张承明, 李峰, 韩颖娟, 孙焕英, 杜漫飞. 基于知识引导的遥感影像融合方法[J]. 自然资源遥感, 2022, 34(2): 47-55.
[11] 于新莉, 宋妍, 杨淼, 黄磊, 张艳杰. 结合空间约束的卷积神经网络多模型多尺度船企场景识别[J]. 自然资源遥感, 2021, 33(4): 72-81.
[12] 刘万军, 高健康, 曲海成, 姜文涛. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3): 97-106.
[13] 韩彦岭, 崔鹏霞, 杨树瑚, 刘业锟, 王静, 张云. 基于残差网络特征融合的高光谱图像分类[J]. 国土资源遥感, 2021, 33(2): 11-19.
[14] 刘钊, 赵桐, 廖斐凡, 李帅, 李海洋. 基于语义分割网络的高分遥感影像城市建成区提取方法研究与对比分析[J]. 国土资源遥感, 2021, 33(1): 45-53.
[15] 仇一帆, 柴登峰. 无人工标注数据的Landsat影像云检测深度学习方法[J]. 国土资源遥感, 2021, 33(1): 102-107.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
版权所有 © 2015 《自然资源遥感》编辑部
地址:北京学院路31号中国国土资源航空物探遥感中心 邮编：100083
电话:010-62060291/62060292 E-mail:zrzyyg@163.com
本系统由北京玛格泰克科技发展有限公司设计开发