Please wait a minute...
 
Remote Sensing for Land & Resources    2019, Vol. 31 Issue (2) : 32-37     DOI: 10.6046/gtzyyg.2019.02.05
|
Application of convolutional neural network feature to remote sensing image registration
Famao YE, Wei LUO, Yanfei SU, Xuqing ZHAO, Hui XIAO, Weidong MIN()
School of Information Engineering, Nanchang University, Nanchang, 330031, China
Download: PDF(5628 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

Successful remote sensing image registration is one of the foundations of many remote sensing applications. Image high-lever features extracted by convolutional neural network (CNN) have achieved excellent performance in image classification and retrieval, and can be used to solve some problems of low-lever image registration features, such as the limitation of expression capability and easily being interfered. Hence, in this paper, the authors investigated the problem as to how to use CNN feature for remote sensing image registration. First, the authors investigated different CNN features from fully connected layers and aggregating convolutional features with different sizes from convolutional layer to register remote sensing image. Then the authors introduced the procedure by using CNN feature for image registration. Finally, the authors compared the registration performances of CNN features and scale-invariant feature transform (SIFT) features after the transformation of the image’s perspective, brightness and scale, respectively. The experimental results show that the CNN feature has better matching performance than the SIFT method in terms of matching accuracy and correct number of corresponding points. The finely tuned CNN feature has stronger robustness to the transformed image than the SIFT feature.

Keywords convolutional neural network      remote sensing image registration      aggregating convolutional features      scale-invariant feature transform (SIFT)     
:  TP79  
Corresponding Authors: Weidong MIN     E-mail: minweidong@ncu.edu.cn
Issue Date: 23 May 2019
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Famao YE
Wei LUO
Yanfei SU
Xuqing ZHAO
Hui XIAO
Weidong MIN
Cite this article:   
Famao YE,Wei LUO,Yanfei SU, et al. Application of convolutional neural network feature to remote sensing image registration[J]. Remote Sensing for Land & Resources, 2019, 31(2): 32-37.
URL:  
https://www.gtzyyg.com/EN/10.6046/gtzyyg.2019.02.05     OR     https://www.gtzyyg.com/EN/Y2019/V31/I2/32
Fig.1  Macro-architecture of AlexNet model
Fig.2  Flow chart of image registration using CNN
Fig.3  Image pairs of bands composition
Fig.4  Image pairs of Landsat TM single band
特征 P-A图像 P-B图像 P-C图像 P-D图像
RMSall RMSLOO Nred RMSall RMSLOO Nred RMSall RMSLOO Nred RMSall RMSLOO Nred
SIFT 0.040 8 0.040 9 64 0.092 1 0.094 1 24 0.070 5 0.070 6 42 0.913 4 1.015 0 7
FC7fine-tuning 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.038 1 0.039 0 49 0.800 2 0.820 2 8
FC6fine-tuning 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.036 2 0.037 1 50 0.662 2 0.839 8 11
Agg1fine-tuning 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.036 2 0.037 1 50 0.698 7 1.077 3 10
Agg2fine-tuning 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.036 2 0.037 1 50 0.670 0 0.714 4 10
Agg3fine-tuning 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.038 1 0.039 0 49 0.737 1 0.758 8 7
Agg4fine-tuning 0.043 1 0.043 3 56 0.109 8 0.111 5 22 0.038 1 0.039 0 49 2.656 5 3.116 7 3
FC7pre-trained 0.038 0 0.038 1 65 0.094 6 0.097 3 25 0.043 3 0.044 4 43 1.893 5 3.635 3 4
FC6pre-trained 0.037 7 0.037 8 71 0.079 6 0.081 3 27 0.042 8 0.043 8 46 1.286 7 1.311 3 5
Agg1pre-trained 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.038 1 0.039 0 49 0.800 2 0.820 2 8
Agg2pre-trained 0.034 2 0.034 2 72 0.079 6 0.081 3 27 0.040 6 0.041 5 48 1.044 4 1.339 5 7
Agg3pre-trained 0.037 9 0.038 0 70 0.079 6 0.081 3 27 0.040 6 0.041 5 48 0.732 0 0.762 7 8
Agg4pre-trained 0.063 8 0.065 6 18 0.263 2 0.261 3 9 0.040 7 0.041 6 46 9.711 4 11.32 4 3
Tab.1  Image registration accuracy of different features
Fig.5  Nred on 4 images pair for various transformations
[1] Zitova B, Flusser J . Image registration methods:A survey[J]. Image and Vision Computing, 2003,21(11):977-1000.
doi: 10.1016/S0262-8856(03)00137-9 url: https://linkinghub.elsevier.com/retrieve/pii/S0262885603001379
[2] Gong M, Zhao S, Jiao L , et al. A novel coarse-to-fine scheme for automatic image registration based on sift and mutual information[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014,52(7):4328-4338.
doi: 10.1109/TGRS.2013.2281391 url: http://ieeexplore.ieee.org/document/6619415/
[3] 张谦, 贾永红, 胡忠文 . 多源遥感影像配准中的SIFT特征匹配改进[J]. 武汉大学学报(信息科学版), 2013,38(4):455-459.
[3] Zhang Q, Jia Y H, Hu Z W . An improved SIFT algorithm for multi-source remote sensing image registration[J]. Geomatics and Information Science of Wuhan University, 2013,38(4):455-459.
[4] 李少毅, 王晓田, 杨开 . 改进的SURF彩色遥感图像配准算法[J]. 计算机测量与控制, 2017,25(1):209-212.
[4] Li S Y, Wang X T, Yang K . An improved SURF algorithm for color remote sensing image registration[J]. Computer Measurement and Control, 2017,25(1):209-212.
[5] Yang K, Karlstrom L, Smith L C , et al. Automated high-resolution satellite image registration using supraglacial rivers on the Greenland ice sheet[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017,99:1-12.
[6] Krizhevsky A, Sutskever I, Hinton G E . ImageNet classification with deep convolutional neural networks[J]. Advance in Neural Information Processing Systems, 2012,25(2):1097-1105.
[7] Simonyan K,Zisserman A.Very deep convolutional networks for large-scale image recognition[EB/OL].( 2015- 04- 10). http://arxiv.org/pdf/1409.1556.pdf.
[8] Chandrasekhar V, Lin J, Morère O , et al. A practical guide to CNNs and fisher vectors for image instance retrieval[J]. Signal Processing, 2015,128:426-439.
[9] 罗建豪, 吴建鑫 . 基于深度卷积特征的细粒度图像分类研究综述[J]. 自动化学报, 2017,43(8):1306-1318.
[9] Luo J H, Wu J X . A survey on fine-grained image categorization using deep convolutional features[J]. Acta Automatica Sinica, 2017,43(8):1306-1318.
[10] 张洪群, 刘雪莹, 杨森 , 等. 深度学习的半监督遥感图像检索[J]. 遥感学报, 2017,21(3):406-414.
[10] Zhang H Q, Liu X Y, Yang S , et al. Retrieval of remote sensing images based on semisupervised deep learning[J]. Journal of Remote Sensing, 2017,21(3):406-414.
[11] 刘峰, 沈同圣, 马新星 . 特征融合的卷积神经网络多波段舰船目标识别[J]. 光学学报, 2017,37(10):248-256.
[11] Liu F, Shen T S, Ma X X . Convolutional neural network based multi-band ship target recognition with feature fusion[J]. Acta Optica Sinica, 2017,37(10):248-256.
[12] Zhu G, Wang Q, Yuan Y , et al. SIFT on manifold:An intrinsic description[J]. Neurocomputing, 2013,113(7):227-233.
doi: 10.1016/j.neucom.2013.01.020 url: https://linkinghub.elsevier.com/retrieve/pii/S0925231213001872
[13] Yosinski J, Clune J, Bengio Y , et al. How transferable are features in deep neural networks?[C]//International Conference on Neural Information Processing Systems. MIT Press, 2014: 3320-3328.
[14] Babenko A,Lempitsky V.Aggregating deep convolutional features for image retrieval[EB/OL].( 2015- 10- 26). http://arxiv.org/pdf/1510.07493v1.pdf.
[15] Wei X S, Luo J H, Wu J , et al. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Transactions on Image Processing, 2017,26(6):2868-2881.
doi: 10.1109/TIP.2017.2688133 url: http://ieeexplore.ieee.org/document/7887720/
[16] Goncalves H, Goncalves J A, Corte-Real L . Measures for an objective evaluation of the geometric correction process quality[J]. IEEE Geoscience and Remote Sensing Letters, 2009,6(2):292-296.
doi: 10.1109/LGRS.2008.2012441 url: http://ieeexplore.ieee.org/document/4770160/
[1] YU Xinli, SONG Yan, YANG Miao, HUANG Lei, ZHANG Yanjie. Multi-model and multi-scale scene recognition of shipbuilding enterprises based on convolutional neural network with spatial constraints[J]. Remote Sensing for Natural Resources, 2021, 33(4): 72-81.
[2] LIU Zhao, ZHAO Tong, LIAO Feifan, LI Shuai, LI Haiyang. Research and comparative analysis on urban built-up area extraction methods from high-resolution remote sensing image based on semantic segmentation network[J]. Remote Sensing for Land & Resources, 2021, 33(1): 45-53.
[3] QIU Yifan, CHAI Dengfeng. A deep learning method for Landsat image cloud detection without manually labeled data[J]. Remote Sensing for Land & Resources, 2021, 33(1): 102-107.
[4] WEI Hongyu, ZHAO Yindi, DONG Jihong. Cooling tower detection based on the improved RetinaNet[J]. Remote Sensing for Land & Resources, 2020, 32(4): 68-73.
[5] LIU Zhao, LIAO Feifan, ZHAO Tong. Remote sensing image urban built-up area extraction and optimization method based on PSPNet[J]. Remote Sensing for Land & Resources, 2020, 32(4): 84-89.
[6] CAI Yaotong, LIU Shutong, LIN Hui, ZHANG Meng. Extraction of paddy rice based on convolutional neural network using multi-source remote sensing data[J]. Remote Sensing for Land & Resources, 2020, 32(4): 97-104.
[7] CAI Zhiling, WENG Qian, YE Shaozhen, JIAN Cairen. Remote sensing image scene classification based on Inception-V3[J]. Remote Sensing for Land & Resources, 2020, 32(3): 80-89.
[8] WU Tong, PENG Ling, HU Yuan. Informal garbage dumps detection in high resolution remote sensing images based on SU-RetinaNet[J]. Remote Sensing for Land & Resources, 2020, 32(3): 90-97.
[9] Yang ZHOU, Yunsheng ZHANG, Siyang CHEN, Zhengrong ZOU, Yaochen ZHU, Ruixue ZHAO. Disaster damage detection in building areas based on DCNN features[J]. Remote Sensing for Land & Resources, 2019, 31(2): 44-50.
[10] Yun GE, Shunliang JIANG, Famao YE, Changlong JIANG, Ying CHEN, Yiling TANG. Aggregating CNN features for remote sensing image retrieval[J]. Remote Sensing for Land & Resources, 2019, 31(1): 49-57.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech