|
Abstract To address the problem that image details are liable to be lost in the process of hyperspectral super-resolution, this study proposed a hyperspectral super-resolution algorithm that combines multi-receptive field features and spectral-spatial attention. By fully using the high- and low-frequency information in hyperspectral images, this algorithm reduces the loss of image details and improves the hyperspectral super-resolution effects. First, in the feature extraction stage, convolution with different sizes of convolutional kernels is used to obtain multi-scale receptive field features. This assists in extracting more high- and low-frequency information from low-resolution images, thus retaining the features of original images. Then, the acquired image features are enhanced by the spatial-spectral attention mechanism, and the reconstruction of spatial-dimension features is conducted using spectral-dimension information. Finally, the features of various groups are fused, and the checkerboard pattern is relieved by applying the pixel deconvolution layer. As a result, clear and high-resolution images can be produced. The proposed super-resolution algorithm that combines multi-receptive field features with spectral-spatial attention was applied to two public datasets Chikusei and Pavia Center Scene, achieving peak signal-to-noise ratios of 39.869 7 and 31.942 2, respectively and structural similarity of 0.937 6 and 0.878 6, respectively. Therefore, the super-resolution algorithm enjoys obvious performance advantages compared to the latest super-resolution algorithms. Overall, the algorithm proposed in this study integrates the advantages of the multi-receptive field feature extraction module and the spatial-spectral attention module and can significantly improve image details.
|
Keywords
hyperspectral image
image super resolution
multi-receptive field feature extraction
attention mechanism
|
|
Corresponding Authors:
WAND Yaxuan
E-mail: quhaicheng@lntu.edu.cn;940556702@qq.com
|
Issue Date: 14 March 2022
|
|
|
[1] |
Muhammad U, Arif M, Ajmal M. Hyperspectral face recognition using 3D-DCT and partial least squares[C]// Burghardt: BMVA Press.UK:University of Bristol,2013:57.1- 57.10.
|
[2] |
Lowe A, Harrison N, French A P. Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress[J]. Plant Methods, 2017, 13(1):80-92.
doi: 10.1186/s13007-017-0233-z
url: http://plantmethods.biomedcentral.com/articles/10.1186/s13007-017-0233-z
|
[3] |
Lin J, Clancy N T, Qi J, et al. Dual-modality endoscopic probe for tissue surface shape reconstruction and hyperspectral imaging enabled by deep neural networks[J]. Medical Image Analysis, 2018, 48:162-176.
doi: 10.1016/j.media.2018.06.004
url: https://linkinghub.elsevier.com/retrieve/pii/S1361841518303736
|
[4] |
Akhtar N, Shafait F, Mian A S. Sparse spatio-spectral representation for hyperspectral image super-resolution[C]// European Conference on Computer Vision, 2014:63-78.
|
[5] |
Dian R, Li S, Guo A, et al. Deep hyperspectral image sharpening[J]. IEEE Transactions on Neural Network Learning Systems, 2018, 29(11):5345-5355.
doi: 10.1109/TNNLS.2018.2798162
url: https://ieeexplore.ieee.org/document/8295275/
|
[6] |
Huang H, Yu J, Sun W. Super-resolution mapping via multi-dictionary based sparse representation[C]// IEEE International Conference on Acoustic,Speech and Signal Processing, 2014:3523-3527.
|
[7] |
练秋生, 张钧芹, 陈书贞. 基于两级字典与分频带字典的图像超分辨率算法[J]. 自动化学报, 2013, 39(8):1310-1320.
|
[7] |
Lian Q S, Zhang J Q, Chen S Z. Image superresolution algorithm based on two-level dictionary and subband dictionary[J]. Acta Automatica Sinica, 2013, 39(8):1310-1320.
doi: 10.3724/SP.J.1004.2013.01310
url: http://www.aas.net.cn/cn/article/doi/10.3724/SP.J.1004.2013.01310
|
[8] |
Yang J, Wright J, Huang T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
doi: 10.1109/TIP.2010.2050625
url: http://ieeexplore.ieee.org/document/5466111/
|
[9] |
Li X, Wei H W, Zhang H Q. Super-resolution reconstruction of single remote sensing image combined with deep learning[J]. Journal of Image and Graphics, 2018, 23(2):209-218.
|
[10] |
Qiao J, Song H, Zhang K, et al. Image super-resolution using conditional generative adversarial network[J]. IET Image Processing, 2019, 13(14):2673-2679.
doi: 10.1049/ipr2.v13.14
url: https://onlinelibrary.wiley.com/toc/17519667/13/14
|
[11] |
Pan J T, Kevin M G, Elisa S, et al. Shallow and deep convolutional networks for saliency prediction[C]// IEEE Conference on Computer Vision and Pattern Recognition, 2016:598-606.
|
[12] |
Li K, Dai D X, Ender K, et al. Hyperspectral image super-resolution with spectral mixup and heterogeneous datasets[EB/OL].(2021-1-19)[2021-3-8].https://arxiv.org/pdf/2101.07589.htm.
url: https://arxiv.org/pdf/2101.07589.htm.
|
[13] |
Aggarwal H K, Majumdar A. Hyperspectral image denoising using spatio-spectral total variation[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(3):442-446.
|
[14] |
Loncan L, De Almeida L B, Briottet X, et al. Hyperspectral pansharpening[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3):27-46.
doi: 10.1109/MGRS.2015.2440094
url: http://ieeexplore.ieee.org/document/7284770/
|
[15] |
Wald L. Data fusion:Definitions and architectures:Fusion of images of different spatial resolutions[M]. Presses des MINES,Computer Science, 2002.
|
[16] |
Peng J Y, Shi C Y, Laugeman E, et al. Implementation of the structural SIMilarity (SSIM) index as a quantitative evaluation tool for dose distribution error detection[J]. Medical Physics, 2020, 47(4):1907-1919.
doi: 10.1002/mp.v47.4
url: https://onlinelibrary.wiley.com/toc/24734209/47/4
|
[17] |
Mei S, Yuan X, Ji J, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network[J]. Remote Sensing, 2017, 9(11):1139.
doi: 10.3390/rs9111139
url: http://www.mdpi.com/2072-4292/9/11/1139
|
[18] |
李成轶, 田淑芳. 基于字典学习的遥感影像超分辨率融合方法[J]. 自然资源遥感, 2017, 29(1):50-56.doi: 10.6046/gtzyyg.2017.01.08.
doi: 10.6046/gtzyyg.2017.01.08
|
[18] |
Li C Y, Tian S F. Super-resolution fusion method for remote sensing image based on dictionary learning[J]. Remote Sensing for Land and Resource, 2017, 29(1):50-56.doi: 10.6046/gtzyyg.2017.01.08.
doi: 10.6046/gtzyyg.2017.01.08
|
[19] |
Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016:1646-1654.
|
[20] |
Yuan Y, Zheng X, Lu X. Hyperspectral image superresolution by transfer learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(5):1963-1974.
doi: 10.1109/JSTARS.4609443
url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4609443
|
[21] |
Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017:136-144.
|
[22] |
Zhang Y, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]// Proceedings of the European Conference on Computer Vision (ECCV), 2018:286-301.
|
[23] |
Dai T, Cai J, Zhang Y, et al. Second-order attention network for single image super-resolution[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2019:11065-11074.
|
[24] |
Mei S, Yuan X, Ji J, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network[J]. Remote Sensing, 2017, 9(11):1139.
doi: 10.3390/rs9111139
url: http://www.mdpi.com/2072-4292/9/11/1139
|
[25] |
Li Y, Zhang L, Dingl C, et al. Single hyperspectral image super-resolution with grouped deep recursive residual network[C]// Proceedings of the IEEE International Conference on Multimedia Big Data, 2018:1-4.
|
[26] |
Sidorov O, Hardeberg J Y. Deep hyperspectral prior:Single-image denoising,inpainting,super-resolution[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop, 2019:3844-3851.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|