Pansharpening algorithm of remote sensing images based on NSCT and PCNN

doi:10.6046/zrzyyg.2022159

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(3482 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

Conventional pansharpening fusion methods suffer inaccurate extraction of details and low spectrum fusion accuracy. This study proposed a pansharpening algorithm of remote sensing images based on nonsubsampled Contourlet transform (NSCT) and pulse coupled neural networks (PCNN) by combining the multi-scale and -directional decomposition characteristics of NSCT and the pulse synchronous emission characteristics of PCNN. The process of this pansharpening algorithm is as follows: first, the details of panchromatic images were extracted through NSCT; then, the extracted detail features were injected into the irregular segmentation regions obtained using the PCNN model; finally, the sharpening fusion results of high-resolution multispectral remote-sensing images were obtained through statistical weighting. As corroborated by the experimental results of WorldView-2 and GF-2 data sets, the pansharpening algorithm outperforms other remote sensing image fusion algorithms in detail preservation and spectral consistency, verifying its effectiveness.

Keywords pulse coupled neural network nonsubsampled Contourlet transform remote sensing image fusion multispectral remote sensing image

ZTFLH:

TP79

Issue Date: 19 September 2023

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Xinyu XU
	Xiaojun LI
	Heting ZHAO
	Junfei GAI

Cite this article:

Xinyu XU,Xiaojun LI,Heting ZHAO, et al. Pansharpening algorithm of remote sensing images based on NSCT and PCNN[J]. Remote Sensing for Natural Resources, 2023, 35(3): 64-70.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2022159 OR https://www.gtzyyg.com/EN/Y2023/V35/I3/64

Fig.1 Schematic diagram of NSCT decomposition

Fig.2 PCNN single neuron model

Fig.3 Sharpening fusion process algorithm in this paper

Fig.4 Pansharpening results of WorldView-2 dataset

Fig.5 Pansharpening results of GF-2 dataset

方法	全分辨率评价指标			降分辨率评价指标
方法	$D λ$	D_S	QNR	Q₄	Q_avg	ERGAS
本文算法	0.012 0	0.067 5	0.921 3	0.691 9	0.680 0	9.587 3
PCA	0.026 7	0.384 6	0.599 0	0.632 6	0.628 3	10.461 9
IHS	0.022 7	0.378 6	0.607 3	0.630 5	0.627 2	10.628 4
GS	0.030 1	0.377 6	0.603 7	0.633 8	0.629 7	10.469 0
GSA	0.054 4	0.392 2	0.574 7	0.653 3	0.641 4	10.695 7
HPF	0.039 4	0.122 5	0.842 9	0.665 9	0.651 2	9.926 1
SFIM	0.029 7	0.099 7	0.873 6	0.670 7	0.656 4	9.856 9
ATWT	0.044 2	0.161 1	0.801 9	0.690 3	0.677 4	9.753 9

Tab.1 WorldView-2 objective evaluation results

方法	全分辨率评价指标			降分辨率评价指标
方法	$D λ$	D_S	QNR	Q₄	Q_avg	ERGAS
本文算法	0.029 7	0.138 6	0.835 9	0.789 3	0.788 9	1.926 6
PCA	0.052 4	0.522 9	0.452 0	0.642 3	0.650 0	2.359 2
IHS	0.054 6	0.525 4	0.448 7	0.636 5	0.644 8	2.371 3
GS	0.052 0	0.522 7	0.452 5	0.642 7	0.650 5	2.356 8
GSA	0.083 3	0.517 9	0.442 0	0.619 7	0.633 5	2.997 4
HPF	0.036 1	0.142 4	0.826 6	0.759 4	0.761 3	1.980 6
SFIM	0.036 3	0.142 7	0.826 2	0.758 4	0.759 5	1.993 1
ATWT	0.040 9	0.197 7	0.769 4	0.774 4	0.776 4	1.966 1

Tab.2 GF-2 objective evaluation results

[1]	杨丽萍, 马孟, 谢巍, 等. 干旱区Landsat8全色与多光谱数据融合算法评价[J]. 国土资源遥感, 2019, 31(4):11-19.doi:10.6046/gtzyyg.2019.04.02. doi: 10.6046/gtzyyg.2019.04.02
[1]	Yang L P, Ma M, Xie W, et al. Fusion algorithm evaluation of Landsat8 panchromatic and multispetral images in arid regions[J]. Remote Sensing for Land and Resources, 2019, 31(4):11-19.doi:10.6046/gtzyyg.2019.04.02. doi: 10.6046/gtzyyg.2019.04.02
[2]	Vivone G, Mura M D, Garzelli A, et al. A new benchmark based on recent advances in multispectral pansharpening:Revisiting pansharpening with classical and emerging pansharpening methods[J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 9(1):53-56. doi: 10.1109/MGRS.6245518 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6245518
[3]	刘万军, 高健康, 曲海成, 等. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372. doi: 10.6046/zrzyyg.2020372
[3]	Liu W J, Gao J K, Qu H C, et al. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372. doi: 10.6046/zrzyyg.2020372
[4]	Li H F, Song D Z, Liu Y, et al. Automatic pavement crack detection by multi-scale image fusion[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(6):2025-2031. doi: 10.1109/TITS.6979 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6979
[5]	宋亚萍. 基于多源影像融合和面向对象的土地利用分类技术研究[D]. 石河子: 石河子大学, 2020.
[5]	Song Y P. Research on land use classification technology based on multi-source image fusion and object-oriented[D]. Shihezi: Shihezi University, 2020.
[6]	Kahraman S, Erturk A. Review and performance comparison of pansharpening algorithms for RASAT images[J]. Istanbul University-Journal of Electrical and Electronics Engineering, 2018, 18(1):109-120. doi: 10.5152/iujeee. url: http://dergipark.gov.tr/iujeee
[7]	Wady S, Bentoutou Y, Bengermikh A, et al. A new IHS and wavelet based pansharpening algorithm for high spatial resolution satellite imagery[J]. Advances in Space Research, 2020, 66(7):1507-1518. doi: 10.1016/j.asr.2020.06.001 url: https://linkinghub.elsevier.com/retrieve/pii/S0273117720304014
[8]	Du C, Gao S. Remote sensing image fusion based on nonlinear IHS and fast nonsubsampled contourlet transform[J]. Journal of the Indian Society of Remote Sensing, 2018, 46(6):2023-2032. doi: 10.1007/s12524-018-0859-y
[9]	Wu Z, Huang Y, Zhang K. Remote sensing image fusion method based on PCA and Curvelet transform[J]. Journal of the Indian Society of Remote Sensing, 2018, 46(5):687-689. doi: 10.1007/s12524-017-0736-0 url: http://link.springer.com/10.1007/s12524-017-0736-0
[10]	张涛, 刘军, 杨可明, 等. 结合Gram-Schmidt变换的高光谱影像谐波分析融合算法[J]. 测绘学报, 2015, 44(9):1042-1047. doi: 10.11947/j.AGCS.2015.20140637
[10]	Zhang T, Liu J, Yang K M, et al. Harmonic analysis fusion algorithm for hyperspectral image combining Gram-Schmidt transform[J]. Journal of Surveying and Mapping, 2015, 44(9):1042-1047.
[11]	Aiazzi B, Baronti S, Selva M. Improving component substitution pansharpening through multivariate regression of MS+Pan data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2007, 45(10):3230-3239. doi: 10.1109/TGRS.2007.901007 url: http://ieeexplore.ieee.org/document/4305344/
[12]	Nunez J, Otazu X, Fors O, et al. Multiresolution-based image fusion with additive wavelet decomposition[J]. IEEE Transactions on Geoscience and Remote Sensing, 1999, 37(3):1204-1211. doi: 10.1109/36.763274 url: http://ieeexplore.ieee.org/document/763274/
[13]	Ma D, Lai H C. Remote sensing image matching based improved ORB in NSCT domain[J]. Journal of the Indian Society of Remote Sensing, 2019, 47(5):801-807. doi: 10.1007/s12524-019-00958-y
[14]	王密, 何鲁晓, 程宇峰, 等. 自适应高斯滤波与SFIM模型相结合的全色多光谱影像融合方法[J]. 测绘学报, 2018, 47(1):82-90. doi: 10.11947/j.AGCS.2018.20170421
[14]	Wang M, He L X, Cheng Y F, et al. Panchromatic multispectral image fusion method based on adaptive Gaussian filter and SFIM model[J]. Journal of Surveying and Mapping, 2018, 47(1):82-90.
[15]	Cunha A L, Zhou J, Do M N, et al. The nonsubsampled Contourlet transform:Theory,design,and applications[J]. IEEE Transactions on Image Processing, 2006, 15(10):3089-3101. doi: 10.1109/TIP.2006.877507 url: http://ieeexplore.ieee.org/document/1703596/
[16]	Eckhorn R, Frien A, Bauer R, et al. High frequency (60-90 Hz) oscillations in primary visual cortex of awake monkey[J]. Neuroreport, 1993, 4(3):243-246. pmid: 8477045
[17]	李凯. 基于PCNN和图像分块融合法的多聚焦图像融合算法研究[D]. 昆明: 云南大学, 2013.
[17]	Li K. Research on multi-focus image fusion algorithm based on PCNN and image block fusion method[D]. Kunming: Yunnan University, 2013.
[18]	Ding S, Zhao X, Hui X, et al. NSCT-PCNN image fusion based on image gradient motivation[J]. IET Computer Vision, 2018, 12(4):377-383. doi: 10.1049/cvi2.v12.4 url: https://ietresearch.onlinelibrary.wiley.com/toc/17519640/12/4
[19]	马冯. 基于NSCT变换的遥感图像融合算法研究[D]. 西安: 长安大学, 2019.
[19]	Ma F. Research on remote sensing image fusion algorithm based on NSCT transform[D]. Xi’an: Chang’an University, 2019.
[20]	闫利, 向天烛. NSCT域内结合边缘特征和自适应PCNN的红外与可见光图像融合[J]. 电子学报, 2016, 44(4):761-766. doi: 10.3969/j.issn.0372-2112.2016.04.002
[20]	Yan L, Xiang T Z. Infrared and visible image fusion based on edge feature and adaptive PCNN in NSCT domain[J]. Journal of Electronics, 2016, 44(4):761-766.

[1]	ZHANG Daming, ZHANG Xueyong, LI Lu, LIU Huayong. Remote sensing image segmentation based on Parzen window density estimation of super-pixels[J]. Remote Sensing for Natural Resources, 2022, 34(1): 53-60.
[2]	MA Ruiqi, CHENG Bo, LIU Xu’nan, LIU Yueming, JIANG Wei, YANG Chen. Research on GF-1 remote sensing IHS image fusion algorithm based on compressed sensing[J]. REMOTE SENSING FOR LAND & RESOURCES, 2017, 29(4): 26-32.
[3]	XIE Bin, YU Le, ZHANG De-Rong. Distributed Fusion Process for Remotely Sensed Images Based on GIS Services Chain[J]. REMOTE SENSING FOR LAND & RESOURCES, 2011, 23(1): 138-142.

Viewed

Full text

Abstract

Cited

Shared

Discussed