A classification network of hyperspectral images with multi-scale feature fusion

doi:10.6046/zrzyyg.2024060

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(5435 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The classification of hyperspectral images faces challenges like ineffective extraction of multi-scale features and easy loss of pose information. Considering these challenges, this study proposed a classification network of hyperspectral images with multi-scale feature fusion-the hierarchical multi-scale concatenation net (HMC-Net). Initially, multi-scale convolution kernels were applied for parallel computing to extract multi-level features. Meanwhile, the 1×1 convolutional kernels were employed to reduce input-output dimensions, balancing computational complexity. These operations enabled efficient feature extraction without significantly increasing the overall computational burden. Subsequently, independent capsule networks were used for parallel processing of features at various scales. The max pooling was improved via dynamic routing to enhance the translation invariance of features, thereby reducing the loss of pose information. Finally, the concatenate operation integrated feature maps of different scales, thereby achieving a precise analysis of multi-level information in the classification of hyperspectral images. Comparative experimental results demonstrate that the HMC-Net achieved an overall accuracy of 94%, 98%, and 99% on the Kennedy Space Center, University of Pavia, and Salinas datasets, respectively. Compared to the latest classification model of hyperspectral images, the HMC-Net exhibited significant performance advantages, validating its effectiveness.

Keywords hyperspectral image multi-scale feature pose information capsule network dynamic routing

ZTFLH:

TP751

Issue Date: 01 July 2025

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Lin WEI
	Haoxiang RAN
	Yuping YIN

Cite this article:

Lin WEI,Haoxiang RAN,Yuping YIN. A classification network of hyperspectral images with multi-scale feature fusion[J]. Remote Sensing for Natural Resources, 2025, 37(3): 113-122.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2024060 OR https://www.gtzyyg.com/EN/Y2025/V37/I3/113

Fig.1 HMC-Net network structure

Fig.2 Data preprocessing

Fig.3 Initialization convolution module

Fig.4 Feature maps of initial convolutional

Fig.5 Capsule layer module

Tab.1 Combination structure of HMC-Net

Tab.2 Results of ablation experiment(%)

Fig.6 Classification results of the ablation experiment

Fig.7 Qualitative comparative experimental results on the Kennedy space center dataset

Tab.3 Quantitative comparative experimental results on the Kennedy space center dataset(%)

Fig.8 Qualitative comparative experimental results on the Pavia university dataset

Tab.4 Quantitative comparative experimental results on the Pavia university dataset(%)

Fig.9-1 Qualitative comparative experimental results on the Salinas dataset

Fig.9-2 Qualitative comparative experimental results on the Salinas dataset

Tab.5 Quantitative comparative experimental results on the Salinas dateset(%)

Fig.10 Comparison chart of model parameter quantity and OA

[1]	Li S T, Song W W, Fang L Y, et al. Deep learning for hyperspectral image classification:An overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9):6690-6709.
[2]	Zhang Y, Wang J, Zhang J. A survey on hyperspectral image processing techniques for environmental monitoring[J]. Remote Sen-sing of Environment, 2023, 258(1): 1-19.
[3]	Liu X, Liang Y, Wang Z. Hyperspectral remote sensing for land cover change detection:A review and meta-analysis[J]. Remote Sensing of Environment, 2023, 258(1): 20-38.
[4]	Landgrebe D. Hyperspectral image data analysis[J]. IEEE Signal Processing Magazine, 2002, 19(1):17-28.
[5]	Qu S M, Li X, Gan Z H. A new hyperspectral image classification method based on spatial-spectral features[J]. Scientific Reports, 2022,12:1541.
[6]	Fukushima K. Neocognitron:A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4):193-202. doi: 10.1007/BF00344251 pmid: 7370364
[7]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[8]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016:770-778.
[9]	Jia Y Q, Szegedy C, Liu W, et al. Going deeper with convolutions[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015:1-9.
[10]	Giri R N, Janghel R R, Pandey S K, et al. Enhanced hyperspectral image classification through pretrained CNN model for robust spatial feature extraction[J]. Journal of Optics, 2023, 53(3):2287-2300.
[11]	Wang A L, Song Y L, Wu H B, et al. A hybrid neural architecture search for hyperspectral image classification[J]. Frontiers in Phy-sics, 2023,11:1159266.
[12]	Chen Y, Zhang J, Liang X, et al. A novel meta-learning-based hyperspectral image classification method based on deep convolutional neural networks and multi-task learning[J]. Frontiers in Physics, 2023, 11(1):1-16.
[13]	Zhang Y, Li J, Zhang L, et al. Hyperspectral image classification method based on 2D-3D CNN and multibranch neural network[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(12):2106-2110.
[14]	Li Z, Huang W, Wang L, et al. A novel convolutional neural network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 11(1):1-16.
[15]	Roy S K, Krishna G, Dubey S R, et al. HybridSN:Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[16]	Hong D F, Han Z, Yao J, et al. SpectralFormer:Rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021,60:5518615.
[17]	Wei L, Ma H Y, Yin Y P, et al. Kmeans-CM algorithm with spectral angle mapper for hyperspectral image classification[J]. IEEE Access, 2023,11:26566-26576.
[18]	Yin Y P, Wei L. Hyperspectral image classification using ensemble extreme learning machine based on fuzzy entropy weights and auto-adapted spatial-spectral features[J]. Multimedia Tools and Applications, 2023, 82(1):217-238.
[19]	Hinton G E, KrizhevskyA, Wang S D. Transforming auto-encoders[C]// Artificial Neural Networks and Machine Learning-ICANN 2011. Springer, 2011:44-51.
[20]	SabourS, FrosstN,Hinton G E. Dynamic routing between capsules[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems.Curran Associates,Inc., 2017:3859-3869.
[21]	Zhang H, Meng L, Wei X, et al. 1D-convolutional capsule network for hyperspectral image classification[C]// Artificial Neural Networks and Machine Learning-ICANN 2019. Springer, 2019:4451-4458.
[22]	Hinton G E, Sabour S, Frosst N. Matrix capsules with EM routing[C]// International Conference on Learning Representations. OpenReview, 2018: 1-15.
[23]	Moraga J, Duzgun H S. JigsawHSI:A network for hyperspectral image classification[J/OL]. arXiv, 2022(2022-06-06). https://arxiv.org/abs/2206.02327. url: https://arxiv.org/abs/2206.02327
[24]	Picco M L, Ruiz M S. Hyperspectral image classification using deep matrix capsules[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 10(1): 1-7.
[25]	Li W, Du Q, Zhang H, et al. A novel deep learning framework for hyperspectral image classification using spatial pyramid pooling[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61(3):1199-12111.
[26]	Zhang Y, Du B. Deep model based transfer and multi-task learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61(8):4759-4770.
[27]	Ahmad M, Khan A M, Mazzara M, et al. A fast and compact 3-D CNN for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020,19:5502205.
[28]	He L, Li H, Plaza A, et al. PlazaA,etal.A novel self-paced learning framework for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 62(4):1738-1752.
[29]	Yu S Q, Jia S, Xu C Y. Convolutional neural networks for hyperspectral image classification[J]. Neurocomputing, 2017,219:88-98.
[30]	Chakraborty T, Trehan U. SpectralNET:Exploring spatial-spectral WaveletCNN for hyperspectral image classification[J/OL]. arXiv, 2021(2021-04-01). https://arxiv.org/abs/2104.00341. url: https://arxiv.org/abs/2104.00341
[31]	Chen S T, Jin M, Ding J. Hyperspectral remote sensing image classification based on dense residual three-dimensional convolutional neural network[J]. Multimedia Tools and Applications, 2021, 80(2):1859-1882.
[32]	Zhang Y, Zhang L, Du B. Hyperspectral image classification based on deep feature fusion and residual learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(2):1576-1590.
[33]	Tun N L, Gavrilov A, Tun N M, et al. Hyperspectral remote sensing images classification using fully convolutional neural network[C]// 2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering. IEEE, 2021:2166-2170.
[34]	Fauvel M, Chanussot J, Benediktsson J A. A spatial-spectral kernel-based approach for the classification of remote-sensing images[J]. Pattern Recognition, 2012, 45(1):381-392.
[35]	Camps-Valls G, Gomez-Chova L, Munoz-Mari J, et al. Composite kernels for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2006, 3(1):93-97.
[36]	Bruzzone L, Carlin L. A multilevel context-based system for classification of very high spatial resolution images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(9):2587-2600.

[1]	CHEN Min, PENG Shuan, WANG Tao, WU Xuefang, LIU Runpu, CHEN Yushuo, FANG Yanru, YANG Pingjian. A comparative study of water body classification of wetlands based on hyperspectral images from the ZY1-02D satellite: A case study of the Baiyangdian wetland[J]. Remote Sensing for Natural Resources, 2025, 37(3): 133-141.
[2]	YE Wujian, XIE Linfeng, LIU Yijun, WEN Xiaozhuo, Li Yang. A MobileNet-based lightweight cloud detection model[J]. Remote Sensing for Natural Resources, 2025, 37(3): 95-103.
[3]	PANG Min. An intelligent platform for extracting patches from multisource domestic satellite images and its application[J]. Remote Sensing for Natural Resources, 2025, 37(2): 148-154.
[4]	ZHANG Yiran, PAN Bin, XU Xia, ZHU Junfeng. Subpixel-level area estimation of green algae based on spectral unmixing in dictionary learning[J]. Remote Sensing for Natural Resources, 2025, 37(2): 88-95.
[5]	HUANG Chuan, LI Yaqin, QI Yueran, WEI Xiaoyan, SHAO Yuanzheng. A hyperspectral unmixing and few-shot classification method based on 3DCAE network[J]. Remote Sensing for Natural Resources, 2025, 37(1): 8-14.
[6]	ZHAO Heting, LI Xiaojun, XU Xinyu, GAI Junfei. An ICM-based adaptive pansharpening algorithm for hyperspectral images[J]. Remote Sensing for Natural Resources, 2024, 36(2): 97-104.
[7]	CAO Delong, LIN Zhen, TANG Tingyuan, LI Chuyu, WANG Xiaorui. Application of 3D-HRNet in Zhuhai-1 OHS hyperspectral images for natural resource monitoring and eco-environment analysis[J]. Remote Sensing for Natural Resources, 2023, 35(4): 34-42.
[8]	LI Lei, SUN Xiyan, JI Yuanfa, FU Wentao. Hyperspectral image classification based on superpixel segmentation and extended multi-attribute profiles[J]. Remote Sensing for Natural Resources, 2023, 35(4): 114-121.
[9]	ZHENG Zongsheng, LIU Haixia, WANG Zhenhua, LU Peng, SHEN Xukun, TANG Pengfei. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 105-111.
[10]	ZHANG Guojian, LIU Shengzhen, SUN Yingjun, YU Kaijie, LIU Lina. Hyperspectral anomaly detection based on the weakly supervised robust autoencoder[J]. Remote Sensing for Natural Resources, 2023, 35(2): 167-175.
[11]	ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
[12]	SUN Xiao, XU Linlin, WANG Xiaoyang, TIAN Ye, WANG Wei, ZHANG Zhongyue. Mineral identification from hyperspectral images based on the optimized K-P-Means unmixing method[J]. Remote Sensing for Natural Resources, 2022, 34(3): 43-49.
[13]	QU Haicheng, WAND Yaxuan, SHEN Lei. Hyperspectral super-resolution combining multi-receptive field features with spectral-spatial attention[J]. Remote Sensing for Natural Resources, 2022, 34(1): 43-52.
[14]	GAO Wenlong, ZHANG Shengwei, LIN Xi, LUO Meng, REN Zhaoyi. The remote sensing-based estimation and spatial-temporal dynamic analysis of SOM in coal mining[J]. Remote Sensing for Natural Resources, 2021, 33(4): 235-242.
[15]	WANG Hua, LI Weiwei, LI Zhigang, CHEN Xueye, SUN Le. Hyperspectral image classification based on multiscale superpixels[J]. Remote Sensing for Natural Resources, 2021, 33(3): 63-71.

Viewed

Full text

Abstract

Cited

Shared

Discussed