Please wait a minute...
 
Remote Sensing for Natural Resources    2025, Vol. 37 Issue (3) : 76-84     DOI: 10.6046/gyzyyg.2023379
|
Pansharpening based on the multiscale weighted neural network in the transform domain
MA Fei(), SUN Lupeng(), YANG Feixia, XU Guangxian
School of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China
Download: PDF(3328 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

To address the issue of spatial and spectral information fusion during pansharpening, this study proposed a sharpening model for panchromatic and multispectral images based on the multiscale weighted pulse-coupled neural network (PCNN) and low-rank and sparse decomposition in the non-subsampled shearlet transform (NSST) domain. The sharpening model consists of low- and high-frequency processing modules. For high-frequency subbands, a method for weighting high-frequency subbands in various scales and directions was proposed, accompanied by an adaptive PCNN model tailored to their characteristics in different directions. In contrast, low-frequency subbands were decomposed into low-rank and sparse parts, with corresponding fusion rules created according to their characteristics. The fused image was then obtained through inverse NSST. The experiments on the sharpening model were conducted using GeoEye,QuickBird, and Pléiades datasets. Moreover, an ablation experiment was designed for the multiscale weighting module for high-frequency information. Compared to suboptimal models, the sharpening model in this study increased the peak signal-to-noise ratio (PSNR) value by approximately 1 dB, 1.6 dB, and 2.2 dB, respectively. The experimental results demonstrate that the sharpening model outperformed other algorithms in index assessment, effectively resolving the challenge of extracting high-frequency information.

Keywords pansharpening      non-subsampled shearlet transform      multiscale weighting      pulse-coupled neural network      low-rank and sparse decomposition     
ZTFLH:  TP79  
Issue Date: 01 July 2025
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Fei MA
Lupeng SUN
Feixia YANG
Guangxian XU
Cite this article:   
Fei MA,Lupeng SUN,Feixia YANG, et al. Pansharpening based on the multiscale weighted neural network in the transform domain[J]. Remote Sensing for Natural Resources, 2025, 37(3): 76-84.
URL:  
https://www.gtzyyg.com/EN/10.6046/gyzyyg.2023379     OR     https://www.gtzyyg.com/EN/Y2025/V37/I3/76
Fig.1  Decomposition diagram of secondary NSST
Fig.2  Algorithm flow chart of this article
Fig.3  Pan-sharpening results on GeoEye dataset
算法 性能指标
PSNR/dB CC UIQI ERGAS SAM RASE RMSE
HPF 38.634 7 0.954 1 0.953 4 1.214 9 0.018 8 4.890 9 3.434 8
DGIF 40.735 7 0.960 3 0.959 3 1.049 2 0.030 6 4.980 0 3.497 3
GF 42.096 4 0.976 7 0.976 3 0.785 6 0.018 3 3.543 6 2.488 6
STEM 40.906 9 0.968 4 0.965 7 1.047 0 0.025 9 4.883 8 3.429 7
PNNet 42.907 1 0.981 8 0.981 3 0.757 7 0.018 0 3.120 5 2.191 4
本文算法 43.848 2 0.984 9 0.984 3 0.684 2 0.017 0 2.868 1 2.014 2
Tab.1  Quality evaluation index of fusion results on GeoEye dataset
Fig.4  Pan-sharpening results on QuickBird dataset
算法 性能指标
PSNR/dB CC UIQI ERGAS SAM RASE RMSE
HPF 25.586 4 0.795 9 0.793 9 6.994 3 0.111 7 26.044 9 14.517 1
DGIF 26.588 3 0.832 0 0.825 4 6.145 2 0.111 3 23.081 5 12.865 3
GF 26.603 2 0.826 9 0.823 6 6.441 5 0.109 9 23.941 7 13.344 8
STEM 26.208 5 0.839 1 0.837 2 6.355 9 0.117 5 24.812 8 13.830 3
PNNet 26.181 1 0.860 0 0.850 2 5.728 1 0.113 3 25.632 7 14.287 3
本文算法 28.130 9 0.878 6 0.862 3 5.0800 0.097 2 20.538 3 11.447 8
Tab.2  Quality evaluation index of fusion result on QuickBird dataset
Fig.5  Pan-sharpening results on Pléiades dataset
算法 性能指标
PSNR/dB CC UIQI ERGAS SAM RASE RMSE
HPF 24.065 9 0.964 0 0.963 3 2.419 9 0.033 9 11.231 8 22.214 3
DGIF 24.727 8 0.964 8 0.964 1 2.240 0 0.054 4 10.991 5 21.739 1
GF 25.186 6 0.962 8 0.961 3 1.845 7 0.023 2 7.795 7 15.418 4
STEM 23.551 1 0.959 1 0.958 1 2.436 5 0.037 3 11.244 2 22.238 8
PNNet 27.351 8 0.980 2 0.978 9 2.015 4 0.045 8 9.964 9 19.708 7
本文算法 29.570 4 0.986 0 0.985 9 1.136 5 0.023 5 4.940 7 9.771 8
Tab.3  Quality evaluation index of fusion resul on Pléiades dataset
算法 耗时/s
GeoEye QuickBird Pléiades
HPF 0.624 1 0.748 4 0.587 3
DGIF 2.364 2 4.596 3 2.113 6
GF 2.012 3 4.218 2 2.423 7
STEM 16.425 2 18.743 9 15.256 2
PNNet 6.236 4 7.240 6 5.254 5
本文算法 15.215 1 20.484 6 13.728 1
Tab.4  Time consumption results of different algorithms on different datasets
数据集 权重模块 PSNR↑/dB CC RASE
GeoEye 43.848 2 0.984 9 2.868 1
× 43.732 8 0.984 5 2.902 9
Quick
Bird
28.130 9 0.868 6 20.538 3
× 28.091 1 0.877 9 20.630 5
Pléiades 29.570 4 0.986 0 4.940 7
× 29.467 4 0.985 9 5.012 3
Tab.5  The results of ablation experiments on three datasets
[1] 刘万军, 高健康, 曲海成, 等. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372.
[1] Liu W J, Gao J K, Qu H C, et al. Ship detection based on multiscale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3):97-106.doi:10.6046/zrzyyg2020372.
[2] Zhong S, Zhang Y, Chen Y, et al. Combining component substitution and multiresolution analysis:A novel generalized BDSD pansharpening algorithm[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(6):2867-2875.
[3] Ghassemian H. A review of remote sensing image fusion methods[J]. Information Fusion, 2016,32:75-89.
[4] Meng X, Shen H, Li H, et al. Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis:Practical discussion and challenges[J]. Information Fusion, 2019,46:102-113.
[5] Zhang K, Zhang F, Wan W, et al. Panchromatic and multispectral image fusion for remote sensing and earth observation:Concepts,taxonomy,literaturereview,evaluation methodologies and challenges ahead[J]. Information Fusion, 2023,93:227-242.
[6] Yuan Q, Wei Y, Meng X, et al. A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3):978-989.
[7] Masi G, Cozzolino D, Verdoliva L, et al. Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8(7):594.
[8] Scarpa G, Vitale S, Cozzolino D. Target-adaptive CNN-based pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(9):5443-5457.
[9] Jin C, Deng L J, Huang T Z, et al. Laplacian pyramid networks:A new approach for multispectral pansharpening[J]. Information Fusion, 2022,78:158-170.
[10] Lei D, Chen P, Zhang L, et al. MCANet:A multidimensional channel attention residual neural network for pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60:1-16.
[11] Su X, Li J, Hua Z. Attention-based and staged iterative networks for pansharpening of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022,60:1-21.
[12] 张杰, 王恒友, 霍连志, 等. 基于特征增强和Three-stream Transformer的高光谱遥感图像全色锐化[J/OL]. 自然资源遥感, 2024, (2024-12-11). https://link.cnki.net/urlid/10.1759.P.20241211.0942.012.
url: https://link.cnki.net/urlid/10.1759.P.20241211.0942.012
[12] Zhang J, Wang H Y, Huo L Z, et al. Pansharpening of hyperspectral remote sensing images based on feature enhancement and three-stream transformer[J/OL]. Remote Sensing for Natural Resources, 2024, (2024-12-11). https://link.cnki.net/urlid/10.1759.P.20241211.0942.012.
url: https://link.cnki.net/urlid/10.1759.P.20241211.0942.012
[13] Yin M, Liu W, Zhao X, et al. A novel image fusion algorithm based on nonsubsampledshearlet transform[J]. Optik, 2014, 125(10):2274-2282.
[14] 徐欣钰, 李小军, 赵鹤婷, 等. NSCT和PCNN 相结合的遥感图像全色锐化算法[J]. 自然资源遥感, 2023, 35(3):64-70.doi:10.6046/zrzyyg.2022159.
[14] Xu X Y, Li X J, Zhao H T, et al. Pansharpening algorithm of remote sensing images based on NSCT and PCNN[J]. Remote Sen-sing for Natural Resources, 2023, 35 (3):64-70.doi:10.6046/zrzyyg.2022159.
[15] Johnson J L. Pulse-coupled neural nets:Translation,rotation,scale,distortion,and intensity signal invariance for images[J]. Applied Optics, 1994, 33(26):6239-6253.
doi: 10.1364/AO.33.006239 pmid: 20936043
[16] Wang X, Bai S, Li Z, et al. The PAN and MS image pansharpening algorithm based on adaptive neural network and sparse representation in the NSST domain[J]. IEEE Access, 2019,7:52508-52521.
[17] 成丽波, 陈鹏宇, 李喆, 等. 基于剪切波变换和拟合优度检验的遥感图像去噪[J]. 吉林大学学报(理学版), 2023, 61(5):1187-1194.
[17] Cheng L B, Chen P Y, Li Z, et al. Remote sensing image denoising based on shearlet transform and goodness of fit test[J]. Journal of Jilin University (Science Edition), 2023, 61(5):1187-1194.
[18] Chen Y, Park S K, Ma Y, et al. A new automatic parameter setting method of a simplified PCNN for image segmentation[J]. IEEE Transactions on Neural Networks, 2011, 22(6):880-892.
doi: 10.1109/TNN.2011.2128880 pmid: 21550882
[19] Gao Q, Zhang P, Xia W, et al. Enhanced tensor RPCA and its application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(6):2133-2140.
[20] Garzelli A, Capobianco L, Alparone L, et al. Hyperspectralpansharpening based on modulation of pixel spectra[C]// 2010 2nd Workshop on Hyperspectral Image and Signal Processing:Evolution in Remote Sensing.IEEE 2nd Workshop on Hyperspectral Image and Signal Processing:Evolution in Remote Sensing. 2010:1-4.
[21] Jian L, Yang X, Wu W, et al. Pansharpening using a guided image filter based on dual-scale detail extraction[J]. Journal of Ambient Intelligence and Humanized Computing, 2018,1-15.
[22] Li Q, Yang X, Wu W, et al. Pansharpening multispectral remote-sensing images with guided filter for monitoring impact of human behavior on environment[J]. Concurrency and Computation:Practice and Experience, 2021, 33(4):e5074.
[23] Tao J, Song C, Song D, et al. Pan-sharpening framework based on multiscale entropy level matching and its application[J]. IEEE Transactions on Geoscience and Remote Sensing. 2022,60:1-21.
[1] GUO Penghao, QIU Jianlin, ZHAO Shunan. A pansharpening algorithm for remote sensing images based on high-frequency domain and multi-depth dilated network[J]. Remote Sensing for Natural Resources, 2024, 36(3): 146-153.
[2] ZHAO Heting, LI Xiaojun, XU Xinyu, GAI Junfei. An ICM-based adaptive pansharpening algorithm for hyperspectral images[J]. Remote Sensing for Natural Resources, 2024, 36(2): 97-104.
[3] HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech