A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images
LYU Yanan1(), ZHU Hong1(), MENG Jian2, CUI Chengling3, SONG Qiqi1
1. College of Ecology Environment, Institute of Disaster Prevention, Langfang 065201, China 2. College of Earth Sciences, Institute of Disaster Prevention, Langfang 065201, China 3. Beijing Geoway Information-Technology Co., Ltd., Beijing 100043, China
Vehicle detection is a hot research topic in the fields of computer vision, photogrammetry, and remote sensing. With the continuous development of deep learning technology, vehicle detection based on remote sensing images has been applied in fields such as smart city construction and intelligent transportation. This study systematically summarized existent vehicle detection algorithms based on remote sensing images and deep learning models and highlighted the classification, analysis, and comparison of one-stage and two-stage vehicle detection algorithms. Moreover, this study summarized the key technologies of vehicle detection in large-scale and complex backgrounds and analyzed the advantages and disadvantages of mainstream deep learning models of vehicle detection based on remote sensing images. Experiments were conducted to evaluate the YOLOv5, Faster-RCNN, FCOS, and SSD algorithms using DOTA and DIOR datasets. The vehicle detection precision based on the DOTA dataset was 0.695, 0.410, 0.370, and 0.251, respectively and that based on the DIOR dataset was 0.566, 0.243, 0.231, and 0.154, respectively. The experimental results show that the small target scale is still the main factor restricting the vehicle detection performance based on remote sensing images and that the application of deep learning models to the detection of small targets is to be further improved. Finally, based on public datasets and the analysis of existing algorithms, this study proposed the solution and development trend of vehicle detection based on remote sensing images in large-scale and complex backgrounds.
吕雅楠, 朱红, 孟健, 崔成玲, 宋其淇. 面向高分辨率遥感影像车辆检测的深度学习模型综述及适应性研究[J]. 自然资源遥感, 2022, 34(4): 22-32.
LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images. Remote Sensing for Natural Resources, 2022, 34(4): 22-32.
Liu T Y, Li W G, Guan J H. Deep learning based object detection in optical remote sensing images:A survey[J]. Radio Communications Technology, 2020, 46(6):624-634.
[3]
Cheng G, Han J. A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117:11-28.
doi: 10.1016/j.isprsjprs.2016.03.014
Cheng Z, Lyu J G, Bai Y Q, et al. High-resolution remote sensing image object detection algorithm combining RPN network and SSD algorithm[J]. Science of Surveying and Mapping, 2021, 46(4):75-82,99.
[5]
Alam M, Wang J F, Cong G, et al. Convolutional neural network for the semantic segmentation of remote sensing images[J]. Mobile Networks and Applications, 2021, 26:200-215.
doi: 10.1007/s11036-020-01703-3
[6]
Ji H, Gao Z, Mei T, et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4):676-680.
doi: 10.1109/LGRS.2019.2930308
Zhang Z, Yao G Y, Li X C, et al. Small target vehicle detection based on improved Faster-RCNN algorithm[J]. Science and Technology Innovation and Application, 2021(4):28-32.
[8]
Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2014:580-587.
Nan X H, Ding L. Overview of typical object detection algorithms based on deep learning[J]. Application Research of Computers, 2020(s2):15-21.
[10]
Girshick R. Fast R-CNN[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2015:1440-1448.
[11]
Ren S, He K, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
doi: 10.1109/TPAMI.2016.2577031
[12]
罗峰. 基于超分辨率迁移学习的遥感图像车辆检测[D]. 厦门: 厦门大学, 2017.
Luo F. Vehicle detection in remote sensing images based on super-resolution transfer learning[D]. Xiamen: Xiamen University, 2017.
[13]
Deng Z P, Hao S, Zhou S L, et al. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3652-3664.
doi: 10.1109/JSTARS.2017.2694890
[14]
Liu K, Mattyus G. Fast multiclass vehicle detection on aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(9):1938-1942.
doi: 10.1109/LGRS.2015.2439517
Wang X, Sui L C, Li D M, et al. Regional convolutional neural network for vehicle detection in remote sensing images[J]. Journal of Electronics and Information Technology, 2018, 35(3):103-108.
Gao X, Li H, Zhang Y, et al. Vehicle detection in remote sensing images of dense areas based on deformable convolution neural network[J]. Journal of Electronics and Information Technology, 2018, 40(12):2812-2819.
[17]
阳理理. 基于人工神经网络的遥感图像车辆检测[D]. 南宁: 广西大学, 2018.
Yang L L. Vehicle detection based on artificial neural network in remote sensing images[D]. Nanning: Guangxi University, 2018.
[18]
孙秉义. 基于遥感图像处理的交通量检测与分析[D]. 上海: 上海交通大学, 2019.
Sun B Y. Traffic volume detection and analysis based on remote sensing images processing[D]. Shanghai: Shanghai Jiaotong University, 2019.
[19]
Xia G S, Bai X, Ding J, et al. DOTA:A large-scale dataset for object detection in aerial images[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2018:3974-3983.
[20]
黄国捷. 基于深度学习的遥感图像车辆目标检测[D]. 苏州: 苏州大学, 2019.
Huang G J. Vehicle target detection from remote sensing images based on deep learning[D]. Suzhou: Soochow University, 2019.
[21]
Ji H, Gao Z, Mei T, et al. Improved Faster R-CNN with multiscale feature fusion and homography augmentation for vehicle detection in remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(11):1761-1765.
doi: 10.1109/LGRS.2019.2909541
[22]
Rottensteiner F, Sohn G, Jung J, et al. The ISPRS benchmark on urban object classification and 3D building reconstruction[J]. ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences, 2012,1-3:293-298.
[23]
Razakarivony S, Jurie F. Vehicle detection in aerial imagery:A small target detection benchmark[J]. Journal of Visual Communication and Image Representation, 2016, 34:187-203.
doi: 10.1016/j.jvcir.2015.11.002
[24]
Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651.
doi: 10.1109/TPAMI.2016.2572683
Liang Z H, Li X, Deng P, et al. Remote sensing images change detection fusion method integrating multi-scale feature attention[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5):668-676.
[26]
Yang X, Sun H, Sun X, et al. Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J]. IEEE Access, 2018, 6:50839-50849.
doi: 10.1109/ACCESS.2018.2869884
[27]
Fu Y, Wu F, Zhao J. Context-Aware and depth wise-based detection on orbit for remote sensing image[C]// 2018 24th International Conference on Pattern Recognition(ICPR).IEEE, 2018:1725-1730.
[28]
Li Q, Mou L, Xu Q, et al. R3-Net:A deep network for multioriented vehicle detection in aerial images and videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7):5028-5042.
doi: 10.1109/TGRS.2019.2895362
[29]
Zhang Z H, Guo W W, Zhu S N, et al. Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(11):1745-1749.
doi: 10.1109/LGRS.2018.2856921
[30]
林钊. 基于深度学习的遥感图像舰船目标检测与识别[D]. 长沙: 国防科技大学, 2018.
Lin Z. Ship detection and recognition in remote sensing images based on deep learning[D]. Changsha: National University of Defense Technology, 2018.
Liu W J, Gao J K, Qu H C, et al. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372.
doi: 10.6046/zrzyyg.2020372
Xu D G, Wang L, Li F. A review of typical object detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2021, 57(8):10-25.
[33]
Redmon J, Divvala S, Girshick R, et al. You only look once:Unified,real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2016:779-788.
[34]
Redmon J, Farhadi A. YOLO9000:Better,faster,stronger[C]// IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2017:6517-6525.
[35]
Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL].(2019-12-25)[2021-12-29]. https://arxiv.org/abs/1804.02767 .
[36]
Bochkovskiy A, Wang C Y, Liao H. YOLOv4:Optimal speed and accuracy of object detection[EB/OL].(2020-04-23)[2021-12/29]. https://arxiv.org/abs/2004.10934 .
[37]
Liu W, Anguelov D, Erhan D, et al. SSD:Single shot multibox detector[J]. Springer,Cham, 2016, 9905:21-37.
[38]
Fu C, Liu W, Ranga A, et al. DSSD:Deconvolutional single shot detector[EB/OL].(2017-01-23)[2021-12/29]. https://arxiv.org/abs/1701.06659v1 .
[39]
Tian Z, Shen C H, Chen H, et al. FCOS:Fully convolutional one-stage object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE, 2020:9626-9635.
Li S H, Shao F J. Vehicle detection model of light weight remote sensing image based on deep learning[J]. Industrial Control Computer, 2020, 33(6):66-69.
[41]
Etten A V. You only look twice:Rapid multi-scale object detection in satellite imagery[EB/OL].(2018-05-24)[2021-12-29]. https://arxiv.org/abs/1805.09512 .
Peng X Y, Zhang W M, Zhong R F. GF-2 satellite image vehicle detection based on improved YOLOv3 model[J]. Science of Surveying and Mapping, 2021, 46(12):147-154.
Tang T Y. Deep convolutional neural network based vehicle detection methods on high resolution optical remote sensing images[D]. Changsha: National University of Defense Technology, 2017.
Hou T, Jiang Y. Resrarch of improved YOLOv4 in remote sensing aircraft target detection[J]. Computer Engineering and Applications, 2021, 57(12):224-230.
Zhao P F, Xie L B, Peng L. A deep small target detection algorithm based on attention mechanism[J]. Computer Science and Technolo-gy, 2022, 16(4):927-937.
Wang M Y, Wang J T, Liu C. Detection of rotating targets in remote sensing images based on key points[J]. Journal of Electronic Measurement and Instrument, 2021, 35(6):102-108.
Tang J Y, Tang C H. Remote sensing image target detection algorithm based on rotating frame and attention mechanism[J]. Electronic Measurement Technique, 2021, 44(13):114-120.
Chen J. Research on maritime target fusion detection in multi-source remote sensing images based on R-YOLO[D]. Wuhan: Huazhong University of Science and Technology, 2019.
Yang Z P, Ding S, Zhang L, et al. An arbitrary angle dense target detection method for remote sensing images without anchor points[J]. Computer Application, 2022, 42(6):1965-1971.
Zhang H Q, Ban Y M, Guo L L, et al. Remote sensing images ship detection method based on YOLOv5[J]. Electronic Measurement Technology, 2021, 44(8):87-92.
Zhang Y L. Research on intelligent detection and recognition metho-ds of ship targets on the sea surface in optical images[D]. Changchun: University of Chinese Academy of Sciences (Changchun Institute of Optics,Fine Mechanics and Physics,CAS), 2021.
[53]
Li K, Wan G, Cheng G, et al. Object detection in optical remote sensing images:A survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307.
doi: 10.1016/j.isprsjprs.2019.11.023