Classification of hyperspectral image based on feature fusion of residual network

doi:10.6046/gtzyyg.2020209

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(4576 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Deep learning technology provides technical means for hyperspectral image classification due to its unique advantages in deep mining of features. However, in the pixel-level feature classification of hyperspectral images, the number of deep learning layers is limited due to the influence of the sample input size, and the depth features in the hyperspectral images cannot be fully mined. The classification of hyperspectral image based on feature fusion of residual network is proposed in this paper. First, the principal component analysis (PCA) method is used to extract the first principal component in the original hyperspectral image, and the residual network is used to effectively extract the spatial spectrum features of the ground objects; then the feature map is expanded by the deconvolution algorithm, and after deconvolution, features of different dimensions are fused with multi-scale features to fully mine the depth feature information in the hyperspectral image, thus further improving the classification accuracy of the hyperspectral image. The ground feature classification experiment was conducted on the two areas of Taihu Lake in Jiangsu and Chaohu Lake in Anhui captured by the “Zhuhai-1” satellite. The results show that, compared with other methods, this method can effectively solve the problem of insufficient depth feature extraction in hyperspectral image classification, thus showing better classification performance.

Keywords deconvolution feature fusion residual network hyperspectral image classification

ZTFLH:

TP79

Corresponding Authors: YANG Shuhu E-mail: ylhan@shou.edu.cn;shyang@shou.edu.cn

Issue Date: 21 July 2021

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Yanling HAN
	Pengxia CUI
	Shuhu YANG
	Yekun LIU
	Jing WANG
	Yun ZHANG

Cite this article:

Yanling HAN,Pengxia CUI,Shuhu YANG, et al. Classification of hyperspectral image based on feature fusion of residual network[J]. Remote Sensing for Land & Resources, 2021, 33(2): 11-19.

URL:

https://www.gtzyyg.com/EN/10.6046/gtzyyg.2020209 OR https://www.gtzyyg.com/EN/Y2021/V33/I2/11

Fig.1 Overall framework of this paper

Fig.2 Conceptual diagram of the residual network

Fig.3 Improved residual network structure

Fig.4 Convolution and deconvolution

Fig.5 Multi-scale fusion structure diagram

Fig.6 B14(R), B7(G), B2(B) false color composite image of Zhuhai-1

Fig.7 Spectral curves of study areas

Tab.1 Number of data set samples(个)

层数	输出尺寸大小	卷积核大小及个数	步长	激活层
conv1i	27×27	$3 × 3 32 3 × 3 32 3 × 3 32$	1×1	ReLU
pool1	14×14	—	2×2	—
conv2i	14×14	$3 × 3 64 3 × 3 64 3 × 3 64$	1×1	ReLU
pool2	7×7	—	2×2	—
Dconv1	18×18	5×5 64	1×1	ReLU
Dconv2	9×9	3×3 64	1×1	ReLU
M1	18×18	32	—	—
conv3i	18×18	$3 × 3 32 3 × 3 32 3 × 3 32$	1×1	ReLU
pool1	9×9	—	2×2	ReLU
conv4i	9×9	$3 × 3 64 3 × 3 64 3 × 3 64$	1×1	ReLU
pool2	5×5	—	2×2	ReLU
conv5	4×4	3×3 128	1×1	ReLU
pool5	2×2	—	2×2	—

Tab.2 Improved residual network

Tab.3 CNN network structure

Tab.4 Traditional residual network

模型	太湖		巢湖
模型	OA/%	Kappa×100	OA/%	Kappa×100
孪生网络	61.27 $±$ 0.86	52.12 $±$ 0.15	64.57 $±$ 0.59	53.48 $±$ 0.25
SVM	76.73	69.02	80.40	70.52
CNN	86.23 $±$ 0.92	79.44 $±$ 1.47	83.25 $±$ 1.22	75.08 $±$ 2.47
GLCMCNN	88.94 $±$ 0.26	80.23 $±$ 0.33	85.32 $±$ 0.46	78.23 $±$ 0.7
传统残差网络	90.07 $±$ 1.88	85.41 $±$ 2.83	89.26 $±$ 1.18	84.18 $±$ 1.56
本文方法	92.27 $±$ 2.45	86.60 $±$ 4.00	91.31 $±$ 3.07	87.14 $±$ 4.64

Tab.5 Comparison results of different methods

Fig.8 Visualization results

Tab.6 Classification results of Taihu Lake data under different numbers of convolution kernels

Tab.7 Classification results of Chaohu Lake data with different numbers of convolution kernels

Tab.8 Classification results of Taihu Lake under different input sizes

Tab.9 Classification results of Chaohu Lake with different input sizes

[1]	李秉璇, 周冰, 贺宣, 等. 针对高光谱图像的目标分类方法现状与展望[J]. 激光与红外, 2020, 50(3):259-265.
[1]	Li B X, Zhou B, He X, et al. Status and prospects of target classification methods for hyperspectral images[J]. Laser and Infrared, 2020, 50(3):259-265.
[2]	张亮. 基于PCA和SVM的高光谱遥感图像分类研究[J]. 光学技术, 2008, 34(s1):184-187.
[2]	Zhang L. Research on classification of hyperspectral remote sensing images based on PCA and SVM[J]. Optical Technology, 2008, 34(s1):184-187.
[3]	芦国军, 陈丽芳. 基于深度卷积神经网络的遥感图像场景分类[J]. 太原师范学院学报(自然科学版), 2019, 18(1):57-62.
[3]	Lu G J, Chen L F. Remote sensing image scene classification based on deep convolutional neural network[J]. Journal of Taiyuan Teachers College (Natural Science Edition), 2019, 18(1):57-62.
[4]	王云艳, 罗冷坤, 周志刚. 基于反卷积高层特征的遥感地物图像分类[J]. 计算机工程与应用, 2020, 56(11):200-206.
[4]	Wang Y Y, Luo L K, Zhou Z G. Remote sensing object image classification based on deconvolution high-level features[J]. Computer Engineering and Applications, 2020, 56(11):200-206.
[5]	于佩鑫, 周询, 刘素红, 等. 东北黑土区侵蚀沟遥感影像特征提取与识别[J]. 遥感学报, 2018, 22(4):611-620.
[5]	Yu P X, Zhou X, Liu S H, et al. Feature extraction and recognition of remote sensing image of eroded gully in black soil region of northeast China[J]. Journal of Remote Sensing, 2018, 22(4):611-620.
[6]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 6:770-778.
[7]	孟佳佳, 王弢. 基于深度残差网络的遥感数据分类[J]. 数字技术与应用, 2019, 37(1):99-101.
[7]	Meng J J, Wang T. Remote sensing data classification based on deep residual network[J]. Digital Technology and Application, 2019, 37(1):99-101.
[8]	张怡卓, 徐苗苗, 王小虎, 等. 残差网络分层融合的高光谱地物分类[J]. 光谱学与光谱分析, 2019, 39(11):3501-3507.
[8]	Zhang Y Z, Xu M M, Wang X H, et al. Classification of hyperspectral features by hierarchical fusion of residual networks[J]. Spectroscopy and Spectral Analysis, 2019, 39(11):3501-3507.
[9]	周俊宇, 赵艳明. 卷积神经网络在图像分类和目标检测应用综述[J]. 计算机工程与应用, 2017, 53(13):34-41.
[9]	Zhou J Y, Zhao Y M. Overview of the application of convolutional neural networks in image classification and target detection[J]. Computer Engineering and Applications, 2017, 53(13):34-41.
[10]	裴君岩, 刘义海. 基于多尺度感受野扩增融合的遥感目标检测算法[J]. 指挥控制与仿真, 2020, 42(1):34-39
[10]	Pei J Y, Liu Y H. Remote sensing target detection algorithm based on multi-scale receptive field amplification fusion[J]. Command Control and Simulation, 2020, 42(1):34-39.
[11]	Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines[Z]. In ICML. 2010.
[12]	Chen P H, Lin C J, Schölkopf, B. A tutorial on v-support vector machines[J]. Applied Stochastic Models in Business and Industry, 2005, 21(2):111-136. doi: 10.1002/asmb.v21:2 url: http://doi.wiley.com/10.1002/asmb.v21%3A2
[13]	Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2):1-9.
[14]	Su H, Wang Y, Xiao J, et al. Improving MODIS sea ice detectability using gray level co-occurrence matrix texture analysis method:A case study in the Bohai Sea[J]. ISPRS Journal of Photogrammetry & Remote Sensing, 2013, 85(85):13-20.

[1]	NIU Xianghua, HUANG Wei, HUANG Rui, JIANG Sili. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.
[2]	ZHENG Zongsheng, LIU Haixia, WANG Zhenhua, LU Peng, SHEN Xukun, TANG Pengfei. Improved 3D-CNN-based method for surface feature classification using hyperspectral images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 105-111.
[3]	ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information[J]. Remote Sensing for Natural Resources, 2022, 34(3): 27-32.
[4]	JIANG Yanan, ZHANG Xin, ZHANG Chunlei, ZHONG Chengcheng, ZHAO Junfang. Classification of remote sensing images based on multi-scale feature fusion using local binary patterns[J]. Remote Sensing for Natural Resources, 2021, 33(3): 36-44.
[5]	LIU Wanjun, GAO Jiankang, QU Haicheng, JIANG Wentao. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3): 97-106.
[6]	LU Qi, QIN Jun, YAO Xuedong, WU Yanlan, ZHU Haochen. Buildings extraction of GF-2 remote sensing image based on multi-layer perception network[J]. Remote Sensing for Land & Resources, 2021, 33(2): 75-84.
[7]	SUN Ke. Remote sensing image classification based on super pixel and peak density[J]. Remote Sensing for Land & Resources, 2020, 32(4): 41-45.
[8]	Benzuo YAO, Fang HE. Spatial and spectral feature hierarchical fusion for hyperspectral image feature extraction[J]. Remote Sensing for Land & Resources, 2019, 31(3): 59-64.
[9]	Wanjun LIU, Tianhui LI, Haicheng QU. Hyperspectral similar sample classification algorithm based on Fisher criterion and TrAdaboost[J]. Remote Sensing for Land & Resources, 2018, 30(4): 41-48.

Viewed

Full text

Abstract

Cited

Shared

Discussed