基于生成对抗网络的遥感影像色彩一致性方法

doi:10.6046/zrzyyg.2021316

摘要
图/表
参考文献
相关文章
Metrics

全文: PDF(5110 KB)

HTML
输出: BibTeX | EndNote (RIS)

摘要

在遥感成像过程中易在拍摄影像内部、影像与影像之间产生亮度不均匀、色彩不一致的现象,通过人工借助图像处理软件进行色彩调节已经不能满足呈几何级数量增长的遥感影像调色需求,因此提出一种针对土地利用率高的复杂城区地物的融合注意力机制无监督循环一致生成对抗网络(channel attention-cycle generative adversarial networks,CA-CycleGAN)。首先,通过直方图调整和Photoshop等软件手工制作用于色彩参考的样本数据集,选择合适的城区影像数据作为待校正影像样本集,将2部分影像分别进行裁切,得到预处理后的影像样本集; 然后,将处理好的待校正影像集和色彩参考影像集通过CA-CycleGAN中,由于在生成器中加入了注意力机制,因此在生成器与鉴别器相互对抗的训练过程中能够利用注意力特征图将生成的重点分配在重要的区域,提高生成影像效果,得到基于城区影像的色彩校正模型以及色彩校正后的影像图。影像校正效果和损失函数图表明,所提出的方法在循环一致生成对抗网络基础上做出了优化,加入注意力机制的生成对抗网络在调整影像色彩上的综合表现效果优于不加注意力机制的生成对抗网络。相较于传统方法大大减少了色彩校正的时间,对比人工调色增加了影像色彩校正效果的稳定性。证明所提出方法在遥感影像匀色工作中优势较明显,具有较好的应用前景。

	服务

	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	王艺儒
	王光辉
	杨化超
	刘慧杰

关键词 ：遥感, 影像色彩校正, 生成一致对抗网络, 城区卫星影像, 注意力机制

Abstract：

Uneven brightness and inconsistent colors are prone to occur inside and between captured images in the process of remote sensing imaging. However, the manual color conditioning combined with image processing software can no longer meet the color matching demand of geometrically increasing remote sensing images. Given this, this study proposed a kind of unsupervised channel-cycle generative adversarial network (CA-CycleGAN) integrated with the attention mechanism suitable for ground objects in complex urban areas with a high land utilization rate. Firstly, the sample data set used for color reference was manually made through histogram adjustment and Photoshop, and the appropriate urban images were selected as the sample set to be corrected. Then, the two kinds of images were cut respectively to obtain the preprocessed image sample sets. Finally, the preprocessed image set to be corrected and the image set for color reference were processed using the CA-CycleGAN. Because the attention mechanism has been added to the generator, the generated focuses can be distributed into key areas using the attention feature map in the training process of the confrontation between the generator and the discriminator, thus improving the image effects and obtaining the color correction model based on urban images and the images after color correction. Both the image correction effect and the loss function diagram show that the proposed method is optimized based on the CycleGAN and that the comprehensive performance of the CycleGAN integrated with the attention mechanism is better than that without the attention mechanism. Compared to conventional methods, the method proposed in this study greatly reduced the time for color correction and achieved more stable image color correction effects than manual color matching. Therefore, the method proposed in this study enjoys significant advantages in the color dodging of remote sensing images and has a good application prospect.

Key words： remote sensing image color correction CycleGAN urban satellite image attention mechanism

收稿日期: 2021-09-27 出版日期: 2022-09-21

ZTFLH:

P236

通讯作者: 王光辉

作者简介: 王艺儒(1997-),女,硕士研究生,研究方向为图像色彩校正。Email: 1306347915@qq.com。

引用本文:

王艺儒, 王光辉, 杨化超, 刘慧杰. 基于生成对抗网络的遥感影像色彩一致性方法[J]. 自然资源遥感, 2022, 34(3): 65-72.
WANG Yiru, WANG Guanghui, YANG Huachao, LIU Huijie. A method for color consistency of remote sensing images based on generative adversarial networks. Remote Sensing for Natural Resources, 2022, 34(3): 65-72.

链接本文:

https://www.gtzyyg.com/CN/10.6046/zrzyyg.2021316 或 https://www.gtzyyg.com/CN/Y2022/V34/I3/65

Fig.1 注意力引导的CycleGAN原理图

Fig.2 通道注意力结构

Fig.3 生成器和鉴别器网络结构

Tab.1 小区域内影像色彩校正效果对比

Fig.4 不同方法色彩校正后拼接效果

Fig.5 不同方法色彩校正的损失函数

Tab.2 最后一个循环时不同色彩校正方法的损失值

Fig.6 验证数据集色彩校正效果示例

[1]	金淑英, 包更生, 马洪超, 等. 海港海底视频影像的灰度不均匀校正方法的研究[J]. 武汉测绘科技大学学报, 2000, 25(6):531-536.
	Jin S Y, Bao G S, Ma H C, et al. Research on gray nonuniformity correction method of seaport seafloor video image[J]. Geomatics and Information Science of Wuhan University, 2000, 25(6):531-536.
[2]	Kim S J, Kim N Y, Lee B G, et al. Wavelet based uneven illumination compensation for defect detection of flat panel display[C]// Iasted International Conference on Signal Processing,Pattern Reco-gnition,and Applications.ACTA Press, 2007:104-108.
[3]	Tan R T. Visibility in bad weather from a single image[C]// IEEE Conference on Computer Vision and Pattern Recognition,IEEE, 2008:1-8.
[4]	谭攀, 吴绍斐, 苏晓军, 等. 基于影像动态分块与HIS结合的匀光算法[J]. 影像技术, 2014, 26(1):55-56.
	Tan P, Wu S F, Su X J, et al. A smoothing algorithm based on dynamic image segmentation and HIS[J]. Image Technology, 2014, 26(1):55-56.
[5]	张振. 光学遥感影像匀光算法研究[D]. 郑州: 解放军信息工程大学, 2010.
	Zhang Z. Research on dodging methods of optical remotely sensed image[D]. Zhengzhou: Information Engineering University, 2010.
[6]	易磊. 遥感影像色彩一致性处理技术研究[D]. 郑州: 解放军信息工程大学, 2015.
	Yi L. Research on color consistency processing for remote sensing image[D]. Zhengzhou: Information Engineering University, 2015.
[7]	任中杰. 遥感瓦片影像色彩一致性研究[D]. 赣州: 江西理工大学, 2019.
	Ren Z J. Research on color consistency of remote sensing tile image[D]. Ganzhou: Jiangxi University of Science and Technology, 2019.
[8]	杜神斌. 数字正射影像镶嵌中色彩一致性方法研究[D]. 南昌: 东华理工大学, 2018.
	Du S B. Research on color consistency method in digital orthophoto mosaic[D]. Nanchang: East China University of Technology, 2018.
[9]	刘晓龙. 基于影像匹配接边纠正的数字正射影像的镶嵌技术[J]. 遥感学报, 2001, 5(2):104-109.
	Liu X L. Image match and overlapping area correction mosaic technology for digital orthophoto maps[J]. National Remote Sensing Bulletin, 2001, 5(2):104-109.
[10]	王密, 潘俊. 一种数字航空影像的匀光方法[J]. 中国图象图形学报, 2004(6):104-108,127.
	Wang M, Pan J. A method of removing the uneven illumination for digital aerial image[J]. Journal of Image and Graphics, 2004(6):104-108,127.
[11]	刘建涛. 基于彩色空间变换的航空数码影像自动匀光处理研究[D]. 西安: 长安大学, 2008.
	Liu T J. Study on dodging of aerial digital image based on color model transform[D]. Xi’an: Chang’an University, 2008.
[12]	李治江. 彩色影像色调重建的理论与实践[D]. 武汉: 武汉大学, 2005.
	Li Z J. Theory and practice on tone reproduction of color photos[D]. Wuhan: Wuhan University, 2005.
[13]	Stumpfel J, Jones A, Wenger A, et al. Direct HDR capture of the sun and sky[C]// Proceedings of the 3rd International Conference on Computer Graphics,Virtual Reality,Visualization and Interaction in Africa, 2004:145-149.
[14]	张振, 朱宝山, 朱述龙. 反差一致性改进的MASK匀光算法[J]. 测绘科学技术学报, 2010(1):54-56.
	Zhang Z, Zhu B S, Zhu S L. An improved MASK dodging method based on contrast consistency processing[J]. Journal of Geomatics Science and Technology, 2010(1):54-56.
[15]	朱述龙, 朱宝山, 王红卫. 遥感影像处理与应用[M]. 北京: 科学出版社, 2006.
	Zhu S L, Zhu B S, Wang H W. Remote sensing image processing and application[M]. Beijing: Science Press, 2006.
[16]	Orsini G, Ramponi G, Carrai P, et al. A modified retinex for image contrast enhancement and dynamics control[C]// International Conference on Image Processing.IEEE, 2003:III-393-396.
[17]	Lam E Y. Combining gray world and retinex theory for automatic white balance in digital photography[C]// International Symposium on Consumer Electronics.IEEE, 2005:134-139.
[18]	Nnolim U, Lee P. Homomorphic filtering of colour images using a spatial filter kernel in the HIS colour space[J]. 2008, 71(2):1738-1743.
[19]	Seow M J, Asari V K. Ratio rule and homomorphic filter for enhancement of digital colour image[J]. Neurocomputing, 2006, 69(7-9):954-958. doi: 10.1016/j.neucom.2005.07.003
[20]	Xue Z X. Semi-supervised convolutional generative adversarial network for hyperspectral image classification[J]. IET Image Processing, 2020, 14(4):709-719. doi: 10.1049/iet-ipr.2019.0869
[21]	李雪, 张力, 王庆栋, 等. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报, 2020, 49(11):1473-1484.
	Li X, Zhang L, Wang Q D, et al. Multi-temporal remote sensing imagery semantic segmentation color consistency adversarial network[J]. Acta Geodaetica Cartographica Sinica, 2020, 49(11):1473-1484.
[22]	王照乾, 孔韦韦, 滕金保, 等. DenseNet生成对抗网络低照度图像增强方法[J]. 计算机工程与应用, 2020, 58(8):214-220.
	Wang Z Q, Kong W W, Teng J B, et al. Low illumination image enhancement method based on DenseNet GAN[J]. Computer Engineering and Applications, 2020, 58(8):214-220.
[23]	Lyu N, Ma H X, Chen C, et al. Remote sensing data augmentation through adversarial training[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14:9318-9333. doi: 10.1109/JSTARS.2021.3110842
[24]	何鎏一. 基于深度学习的去雾算法设计[D]. 青岛: 青岛大学, 2020.
	He L Y. Design of defogging algorithm based on deep learning[D]. Qingdao: Qingdao University, 2020.
[25]	Hu A, Xie Z, Xu Y, et al. Unsupervised haze removal for high-resolution optical remote-sensing images based on improved generative adversarial networks[J]. Remote Sensing, 2020, 12(24):4162. doi: 10.3390/rs12244162
[26]	冀俭俭. 基于生成对抗网络的分级联合图像补全方法[D]. 北京: 北京林业大学, 2020.
	Ji J J. Hierarchical jiont image completion method based on generative adversarial network[D]. Beijing: Beijing Forestry University, 2020.
[27]	郑凯, 李建胜, 王俊强, 等. DCLS-GAN:利用生成对抗网络的天绘一号卫星高原地区影像去云方法[J]. 测绘学报, 2021, 50(2):248-259.
	Zheng K, Li J S, Wang J Q, et al. DCLS-GAN:Cloud removal method for plateau area of TH-1 satellite image[J]. Acta Geodaetica Cartographica Sinica, 2021, 50(2):248-259.
[28]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// 2017 IEEE International Conference on Computer Vision.IEEE, 2017,17453078.

[1]	牛祥华, 黄微, 黄睿, 蒋斯立. 基于注意力特征融合的高保真遥感图像薄云去除[J]. 自然资源遥感, 2023, 35(3): 116-123.
[2]	董婷, 符潍奇, 邵攀, 高利鹏, 武昌东. 基于改进全连接条件随机场的SAR影像变化检测[J]. 自然资源遥感, 2023, 35(3): 134-144.
[3]	王建强, 邹朝晖, 刘荣波, 刘志松. 基于U²-Net深度学习模型的沿海水产养殖塘遥感信息提取[J]. 自然资源遥感, 2023, 35(3): 17-24.
[4]	唐晖, 邹娟, 尹向红, 余姝辰, 贺秋华, 赵动, 邹聪, 罗建强. 基于高分遥感的洞庭湖区河湖采砂监管及典型案例分析[J]. 自然资源遥感, 2023, 35(3): 302-309.
[5]	于航, 安娜, 汪洁, 邢宇, 许文佳, 步凡, 王晓红, 杨金中. 黔西南采煤塌陷区高分遥感动态监测——以六盘水市煤矿采空塌陷区为例[J]. 自然资源遥感, 2023, 35(3): 310-318.
[6]	王静, 王佳, 徐江琪, 黄邵东, 刘东云. 改进遥感生态指数的典型海岸带城市生态环境质量评价——以湛江市为例[J]. 自然资源遥感, 2023, 35(3): 43-52.
[7]	徐欣钰, 李小军, 赵鹤婷, 盖钧飞. NSCT和PCNN相结合的遥感图像全色锐化算法[J]. 自然资源遥感, 2023, 35(3): 64-70.
[8]	刘立, 董先敏, 刘娟. 顾及地学特征的遥感影像语义分割模型性能评价方法[J]. 自然资源遥感, 2023, 35(3): 80-87.
[9]	赵海岚, 蒙继华, 纪云鹏. 遥感技术在苹果园精准种植管理中的应用现状及展望[J]. 自然资源遥感, 2023, 35(2): 1-15.
[10]	郑宗生, 刘海霞, 王振华, 卢鹏, 沈绪坤, 唐鹏飞. 改进3D-CNN的高光谱图像地物分类方法[J]. 自然资源遥感, 2023, 35(2): 105-111.
[11]	熊东阳, 张林, 李国庆. 基于最大熵模型的遥感土地利用多分类研究[J]. 自然资源遥感, 2023, 35(2): 140-148.
[12]	王海雯, 贾俊青, 李北辰, 董永平, 哈斯尔. 基于遥感和行业调查数据的城市土地集约利用水平评估[J]. 自然资源遥感, 2023, 35(2): 149-156.
[13]	李和谋, 白娟, 甘甫平, 李贤庆, 王泽坤. 遥感估算河道流量研究进展[J]. 自然资源遥感, 2023, 35(2): 16-24.
[14]	郭晓萌, 方秀琴, 杨露露, 曹煜. 基于人工神经网络的西辽河流域根区土壤湿度估算[J]. 自然资源遥感, 2023, 35(2): 193-201.
[15]	方贺, 张育慧, 何月, 李正泉, 樊高峰, 徐栋, 张春阳, 贺忠华. 浙江省植被生态质量时空变化及其驱动因素分析[J]. 自然资源遥感, 2023, 35(2): 245-254.

Viewed

Full text

Abstract

Cited

Shared

Discussed