Residual dual regression network for super-resolution reconstruction of remote sensing images

doi:10.6046/zrzyyg.2021208

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(3859 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

In order to solve the problem of poor model generalizing ability in real super-resolution reconstruction of remote sensing images, which is easily caused by the use of artificial high-low resolution image pairs, combined with the residual in residual (RIR) module of residual channel attention network (RCAN), dual regression network (DRN) is improved, and residual dual regression network (RDRN) is proposed. Ten thousand 512 × 512 pixel images from LandCover.ai and DIOR aerial image data sets were selected to form the sample data set for training and testing the network, and the reconstruction results were compared with those of other super-resolution network models. The experimental results show that RDRN has an excellent performance in both reconstruction quality and model parameters. It can achieve a better super segmentation reconstruction effect with lower model complexity and has good generalization ability for different low-resolution remote sensing images.

Keywords remote sensing image super-resolution reconstruction dual regression residual channel attention closed network

ZTFLH:

TP79

Corresponding Authors: LI Jiatian E-mail: sxm.320@qq.com;ljtwcx@163.com

Issue Date: 20 June 2022

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Xiaomei SHANG
	Jiatian LI
	Shaoyun LYU
	Ruchun YANG
	Chao YANG

Cite this article:

Xiaomei SHANG,Jiatian LI,Shaoyun LYU, et al. Residual dual regression network for super-resolution reconstruction of remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(2): 112-120.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2021208 OR https://www.gtzyyg.com/EN/Y2022/V34/I2/112

Fig.1 Dual learning SR reconstruction model

Fig.2 Structure of RIR block

Fig.3 Network structure of RDRN

Fig.4 Overlay-tile strategy

Fig.5 Visual comparison of SR reconstruction at different scales

Fig.6 Visual comparison of different SR methods

Tab.1 Performance comparison of different SR algorithm

Tab.2 Comparison of model parameters

Tab.3 Influence of double regression scheme on SR performance(dB)

Tab.4 Influence of super parameter λ in equation (dB)

Fig.8 Influence of the proportion of unpaired data on reconstructed SR of 4 times

[1]	张艳, 卢宣铭, 刘国瑞, 等. 多路径特征融合的遥感图像超分辨率重建算法[J]. 遥感信息, 2021, 36(2):46-53.
[1]	Zhang Y, Lu X M, Liu G R, et al. Super resolution reconstruction algorithm of remote sensing image based on multi-path feature fusion[J]. Remote Sensing Information, 2021, 36(2):46-53.
[2]	Harris J L. Diffraction and resolving power[J]. Journal of the Optical Society of America, 1964, 54(7):931-936. doi: 10.1364/JOSA.54.000931 url: https://opg.optica.org/abstract.cfm?URI=josa-54-7-931
[3]	苏衡, 周杰, 张志浩. 超分辨率图像重建方法综述[J]. 自动化学报, 2013, 39(8):1202-1213.
[3]	Su H, Zhou J, Zhang Z H. Review of super resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8):1202-1213. doi: 10.3724/SP.J.1004.2013.01202 url: http://www.aas.net.cn/cn/article/doi/10.3724/SP.J.1004.2013.01202
[4]	Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution[C]// David Fleet.Proceedings of European Conference on Computer Vision. Zurich: Computer Vision-ECCV, 2014,184-199.
[5]	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]// IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2016,1646-1654.
[6]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// IEEE.Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016,770-778.
[7]	Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution[C]// IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016:1637-1645.
[8]	Tai Y, Yang J, Liu X. Image super-resolution via deep recursive residual network[C]// IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE, 2017:3147-3155.
[9]	刘帅, 朱亚杰, 薛磊. 一种结合稀疏表示和纹理分块的遥感影像超分辨率方法[J]. 武汉大学学报(信息科学版), 2015, 40(5):578-582.
[9]	Liu S, Zhu Y J, Xue L. A super resolution method for remote sensing image based on sparse representation and texture segmentation[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5):578-582.
[10]	吴琼, 田越, 周春平, 等. 遥感图像超分辨率研究的现状和发展[J]. 测绘科学, 2008, 33(6):66-69,15.
[10]	Wu Q, Tian Y, Zhou C P, et al. Current situation and development of research on super resolution of remote sensing image[J]. Surveying and Mapping Science, 2008, 33(6):66-69,15.
[11]	Shocher A, Cohen N, Irani M. “Zero-shot” super-resolution using deep internal learning[C]// IEEE.Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018:3118-3126.
[12]	Bulat A, Yang J, Tzimiropoulos G. To learn image super-resolution,use a GAN to learn how to do image degradation first[C]// Vittorio Ferrari.Proceedings of the European Conference on Computer Vision).Munich:ECCV, 2018:185-200.
[13]	Yi Z L, Zhang H, Tan P, et al. DualGAN:unsupervised dual learning for image-to-image translation[C]// IEEE.Proceedings of the IEEE International Conference on Computer Vision.Venice:ICCV, 2017:2868-2876.
[14]	Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]// IEEE.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE, 2017:4681-4690.
[15]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J/OL]. https://arxiv.org/abS/1409.1556v3. url: https://arxiv.org/abS/1409.1556v3
[16]	Yuan Y, Liu S, Zhang J, et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks[C]// IEEE. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops: IEEE, 2018:701-710.
[17]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// IEEE.Proceedings of the IEEE International Conference on Computer Vision.Venice:IEEE, 2017:2223-2232.
[18]	陈赛健, 朱远平. 基于生成对抗网络的文本图像联合超分辨率与去模糊方法[J]. 计算机应用, 2020, 40(3):859-864.
[18]	Chen S J, Zhu Y P. Text image joint super resolution and deblurring method based on generative countermeasure network[J]. Computer Applications, 2020, 40(3):859-864.
[19]	李欣, 韦宏卫, 张洪群. 结合深度学习的单幅遥感图像超分辨率重建[J]. 中国图象图形学报, 2018, 23(2):209-218.
[19]	Li X, Wei H W, Zhang H Q. Super resolution reconstruction of single remote sensing image combined with deep learning[J]. Chinese Journal of Image Graphics, 2018, 23(2):209-218.
[20]	Guo Y, Chen J, Wang J, et al. Closed-loop matters:Dual regression networks for single image super-resolution[C]// IEEE.IEEE Conference on Computer Vision and Pattern Recognition.Seattle:CVPR, 2020:5407-5416.
[21]	Zhang Y, Li K, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]// Vittorio Ferrari.Proceedings of the European Conference on Computer Vision.Munich:ECCV, 2018:286-301.
[22]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019:3141-3149.
[23]	Yu Z, Qiang Y. An overview of multi-task learning[J]. National Science Review, 2018, 5(1):30-43. doi: 10.1093/nsr/nwx105 url: https://academic.oup.com/nsr/article/5/1/30/4101432
[24]	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251.
[24]	Zhou F Y, Jin L P, Dong J. Review of convolutional neural networks[J]. Chinese Journal of Computers, 2017, 40(6):1229-1251.
[25]	张顺, 龚怡宏, 王进军. 深度卷积神经网络的发展及其在计算机视觉领域的应用[J]. 计算机学报, 2019, 42(3):453-482.
[25]	Zhang S, Gong Y H, Wang J J. Development of deep convolutional neural network and its application in computer vision[J]. Chinese Journal of Computers, 2019, 42(3):453-482.
[26]	Boguszewski A, Batorski D, Ziemba-Jankowska N, et al. LandCover. ai:Dataset for automatic mapping of buildings,woodlands,water and roads from aerial imagery[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville:IEEE, 2021:1102-1110.
[27]	Li K, Wan G, Cheng G, et al. Object detection in optical remote sensing images:A survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307. doi: 10.1016/j.isprsjprs.2019.11.023 url: https://linkinghub.elsevier.com/retrieve/pii/S0924271619302825
[28]	Wan M, Yazid, et al. A review of image quality assessment (IQA):SNR,GCF,AD,NAE,PSNR,ME[J]. Journal of Advanced Research in Computing and Applications. 2017, 7(1):1-7.

[1]	WANG Jianqiang, ZOU Zhaohui, LIU Rongbo, LIU Zhisong. A method for extracting information on coastal aquacultural ponds from remote sensing images based on a U²-Net deep learning model[J]. Remote Sensing for Natural Resources, 2023, 35(3): 17-24.
[2]	TANG Hui, ZOU Juan, YIN Xianghong, YU Shuchen, HE Qiuhua, ZHAO Dong, ZOU Cong, LUO Jianqiang. River and lake sand mining in the Dongting Lake area: Supervision based on high-resolution remote sensing images and typical case analysis[J]. Remote Sensing for Natural Resources, 2023, 35(3): 302-309.
[3]	XU Xinyu, LI Xiaojun, ZHAO Heting, GAI Junfei. Pansharpening algorithm of remote sensing images based on NSCT and PCNN[J]. Remote Sensing for Natural Resources, 2023, 35(3): 64-70.
[4]	NIU Xianghua, HUANG Wei, HUANG Rui, JIANG Sili. A high-fidelity method for thin cloud removal from remote sensing images based on attentional feature fusion[J]. Remote Sensing for Natural Resources, 2023, 35(3): 116-123.
[5]	ZHANG Xian, LI Wei, CHEN Li, YANG Zhaoying, DOU Baocheng, LI Yu, CHEN Haomin. Research progress and prospect of remote sensing-based feature extraction of opencast mining areas[J]. Remote Sensing for Natural Resources, 2023, 35(2): 25-33.
[6]	DIAO Mingguang, LIU Yong, GUO Ningbo, LI Wenji, JIANG Jikang, WANG Yunxiao. Mask R-CNN-based intelligent identification of sparse woods from remote sensing images[J]. Remote Sensing for Natural Resources, 2023, 35(2): 97-104.
[7]	XIONG Dongyang, ZHANG Lin, LI Guoqing. MaxEnt-based multi-class classification of land use in remote sensing image interpretation[J]. Remote Sensing for Natural Resources, 2023, 35(2): 140-148.
[8]	HU Jianwen, WANG Zeping, HU Pei. A review of pansharpening methods based on deep learning[J]. Remote Sensing for Natural Resources, 2023, 35(1): 1-14.
[9]	ZHAO Linghu, YUAN Xiping, GAN Shu, HU Lin, QIU Mingyu. An information extraction model of roads from high-resolution remote sensing images based on improved Deeplabv3+[J]. Remote Sensing for Natural Resources, 2023, 35(1): 107-114.
[10]	LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(4): 22-32.
[11]	SHEN Jun’ao, MA Mengting, SONG Zhiyuan, LIU Tingzhou, ZHANG Wei. Water information extraction from high-resolution remote sensing images using the deep-learning based semantic segmentation model[J]. Remote Sensing for Natural Resources, 2022, 34(4): 129-135.
[12]	CHENG Tao. A method for vector geographic information acquisition based on synchronous correction with remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(3): 59-64.
[13]	TANG Wenkui, YU Lu, ZHOU Weiqi, YUE Jun, ZHOU Zheng. Dynamic changes in the landscape connectivity in Shenzhen City determined based on the long time series of remote sensing data[J]. Remote Sensing for Natural Resources, 2022, 34(3): 97-105.
[14]	MA Xiaojian, ZHAO Fashun, LIU Yanbin. Multi-feature fusion-based recognition and processing of impulse noise in remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(3): 17-26.
[15]	SU Fu, YU Haipeng, ZHU Weixi. Application of label clustering loss in the classification of remote sensing images[J]. Remote Sensing for Natural Resources, 2022, 34(2): 144-151.

Viewed

Full text

Abstract

Cited

Shared

Discussed