A classification network of hyperspectral images with multi-scale feature fusion
WEI Lin1,2(), RAN Haoxiang1(), YIN Yuping3
1. School of Software, Liaoning Technical University, Huludao 125105, China 2. Department of Basic Education, Liaoning Technical University, Huludao 125105, China 3. School of Electrical and Control Engineering, Liaoning Technical University, Huludao 125105, China
The classification of hyperspectral images faces challenges like ineffective extraction of multi-scale features and easy loss of pose information. Considering these challenges, this study proposed a classification network of hyperspectral images with multi-scale feature fusion-the hierarchical multi-scale concatenation net (HMC-Net). Initially, multi-scale convolution kernels were applied for parallel computing to extract multi-level features. Meanwhile, the 1×1 convolutional kernels were employed to reduce input-output dimensions, balancing computational complexity. These operations enabled efficient feature extraction without significantly increasing the overall computational burden. Subsequently, independent capsule networks were used for parallel processing of features at various scales. The max pooling was improved via dynamic routing to enhance the translation invariance of features, thereby reducing the loss of pose information. Finally, the concatenate operation integrated feature maps of different scales, thereby achieving a precise analysis of multi-level information in the classification of hyperspectral images. Comparative experimental results demonstrate that the HMC-Net achieved an overall accuracy of 94%, 98%, and 99% on the Kennedy Space Center, University of Pavia, and Salinas datasets, respectively. Compared to the latest classification model of hyperspectral images, the HMC-Net exhibited significant performance advantages, validating its effectiveness.
Li S T, Song W W, Fang L Y, et al. Deep learning for hyperspectral image classification:An overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9):6690-6709.
[2]
Zhang Y, Wang J, Zhang J. A survey on hyperspectral image processing techniques for environmental monitoring[J]. Remote Sen-sing of Environment, 2023, 258(1): 1-19.
[3]
Liu X, Liang Y, Wang Z. Hyperspectral remote sensing for land cover change detection:A review and meta-analysis[J]. Remote Sensing of Environment, 2023, 258(1): 20-38.
[4]
Landgrebe D. Hyperspectral image data analysis[J]. IEEE Signal Processing Magazine, 2002, 19(1):17-28.
[5]
Qu S M, Li X, Gan Z H. A new hyperspectral image classification method based on spatial-spectral features[J]. Scientific Reports, 2022,12:1541.
[6]
Fukushima K. Neocognitron:A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4):193-202.
doi: 10.1007/BF00344251
pmid: 7370364
[7]
Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[8]
He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016:770-778.
[9]
Jia Y Q, Szegedy C, Liu W, et al. Going deeper with convolutions[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015:1-9.
[10]
Giri R N, Janghel R R, Pandey S K, et al. Enhanced hyperspectral image classification through pretrained CNN model for robust spatial feature extraction[J]. Journal of Optics, 2023, 53(3):2287-2300.
[11]
Wang A L, Song Y L, Wu H B, et al. A hybrid neural architecture search for hyperspectral image classification[J]. Frontiers in Phy-sics, 2023,11:1159266.
[12]
Chen Y, Zhang J, Liang X, et al. A novel meta-learning-based hyperspectral image classification method based on deep convolutional neural networks and multi-task learning[J]. Frontiers in Physics, 2023, 11(1):1-16.
[13]
Zhang Y, Li J, Zhang L, et al. Hyperspectral image classification method based on 2D-3D CNN and multibranch neural network[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(12):2106-2110.
[14]
Li Z, Huang W, Wang L, et al. A novel convolutional neural network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 11(1):1-16.
[15]
Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[16]
Hong D F, Han Z, Yao J, et al. SpectralFormer:Rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021,60:5518615.
[17]
Wei L, Ma H Y, Yin Y P, et al. Kmeans-CM algorithm with spectral angle mapper for hyperspectral image classification[J]. IEEE Access, 2023,11:26566-26576.
[18]
Yin Y P, Wei L. Hyperspectral image classification using ensemble extreme learning machine based on fuzzy entropy weights and auto-adapted spatial-spectral features[J]. Multimedia Tools and Applications, 2023, 82(1):217-238.
[19]
Hinton G E, KrizhevskyA, Wang S D. Transforming auto-encoders[C]// Artificial Neural Networks and Machine Learning-ICANN 2011. Springer, 2011:44-51.
[20]
SabourS, FrosstN,Hinton G E. Dynamic routing between capsules[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems.Curran Associates,Inc., 2017:3859-3869.
[21]
Zhang H, Meng L, Wei X, et al. 1D-convolutional capsule network for hyperspectral image classification[C]// Artificial Neural Networks and Machine Learning-ICANN 2019. Springer, 2019:4451-4458.
[22]
Hinton G E, Sabour S, Frosst N. Matrix capsules with EM routing[C]// International Conference on Learning Representations. OpenReview, 2018: 1-15.
[23]
Moraga J, Duzgun H S. JigsawHSI:A network for hyperspectral image classification[J/OL]. arXiv, 2022(2022-06-06). https://arxiv.org/abs/2206.02327.
[24]
Picco M L, Ruiz M S. Hyperspectral image classification using deep matrix capsules[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 10(1): 1-7.
[25]
Li W, Du Q, Zhang H, et al. A novel deep learning framework for hyperspectral image classification using spatial pyramid pooling[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61(3):1199-12111.
[26]
Zhang Y, Du B. Deep model based transfer and multi-task learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61(8):4759-4770.
[27]
Ahmad M, Khan A M, Mazzara M, et al. A fast and compact 3-D CNN for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020,19:5502205.
[28]
He L, Li H, Plaza A, et al. PlazaA,etal.A novel self-paced learning framework for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 62(4):1738-1752.
[29]
Yu S Q, Jia S, Xu C Y. Convolutional neural networks for hyperspectral image classification[J]. Neurocomputing, 2017,219:88-98.
[30]
Chakraborty T, Trehan U. SpectralNET:Exploring spatial-spectral WaveletCNN for hyperspectral image classification[J/OL]. arXiv, 2021(2021-04-01). https://arxiv.org/abs/2104.00341.
[31]
Chen S T, Jin M, Ding J. Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network[J]. Multimedia Tools and Applications, 2021, 80(2):1859-1882.
[32]
Zhang Y, Zhang L, Du B. Hyperspectral image classification based on deep feature fusion and residual learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(2):1576-1590.
[33]
Tun N L, Gavrilov A, Tun N M, et al. Hyperspectral remote sensing images classification using fully convolutional neural network[C]// 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering. IEEE, 2021:2166-2170.
[34]
Fauvel M, Chanussot J, Benediktsson J A. A spatial-spectral kernel-based approach for the classification of remote-sensing images[J]. Pattern Recognition, 2012, 45(1):381-392.
[35]
Camps-Valls G, Gomez-Chova L, Munoz-Mari J, et al. Composite kernels for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2006, 3(1):93-97.
[36]
Bruzzone L, Carlin L. A multilevel context-based system for classification of very high spatial resolution images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(9):2587-2600.
2025, Vol. 37 
), 冉浩翔1(
