Please wait a minute...
 
Remote Sensing for Land & Resources    2020, Vol. 32 Issue (4) : 61-67     DOI: 10.6046/gtzyyg.2020.04.09
|
Detection and recognition of road traffic signs in UAV images based on Mask R-CNN
CHEN Pengdi1(), HUANG Liang1,2(), XIA Yan1, YU Xiaona3, GAO Xiaxia1
1. Faculty of Land Resource Engineering, Kunming University of Science and Technology, Kunming 650093, China
2. Surveying and Mapping Geo-Informatics Technology Research Center on Plateau Mountains of Yunnan Higher Education, Kunming 650093, China
3. Kunming Vocational and Technical College of Industry, Kunming 650093, China
Download: PDF(4341 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks    
Abstract  

The detection and recognition of traffic signs is an important part of the intelligent driving navigation system. However, due to the shortcomings of low accuracy, high time complexity and poor robustness, the traditional method cannot meet the current needs of intelligent driving. Therefore, a method for detecting and recognizing road traffic signs of UAV images based on Mask R-CNN is proposed. Firstly, a set of high-quality UAV images road traffic sign data sets are produced. Then, based on the statistics of 200 labeled landmarks features, the region proposal network (RPN) structure anchor boxes width-to-height ratio and initial parameters in Mask R-CNN are improved to make it better applied to UAV images road sign scenes. Finally, the precision-recall (PR) curve and mean average precision (mAP) are used for accuracy evaluation. The experimental results show that the anchor boxes width-to-height ratio is better when the ratio is 1∶1, 1∶2, 1∶3; and that the average detection accuracy obtained by this method is 98.33%, which is higher than the accuracy of Faster R-CNN and YOLOv3, indicating better effectiveness.
Keywords UAV images      traffic sign detection      traffic sign recognition      Mask R-CNN      RPN      anchor boxes     
:  P231  
Corresponding Authors: HUANG Liang     E-mail: cpdhn1058475189@163.com;kmhuangliang@163.com
Issue Date: 23 December 2020
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Pengdi CHEN
Liang HUANG
Yan XIA
Xiaona YU
Xiaxia GAO
Cite this article:   
Pengdi CHEN,Liang HUANG,Yan XIA, et al. Detection and recognition of road traffic signs in UAV images based on Mask R-CNN[J]. Remote Sensing for Land & Resources, 2020, 32(4): 61-67.
URL:  
https://www.gtzyyg.com/EN/10.6046/gtzyyg.2020.04.09     OR     https://www.gtzyyg.com/EN/Y2020/V32/I4/61
Tab.1  Sample dataset
Fig.1  Mask R-CNN network framework
Fig.2  FPN architecture
Fig.3  RPN network architecture
Fig.4  RoI Align bilinear interpolation
Fig.5  Loss rate of Mask R-CNN
Fig.6  PR curve of Mask R-CNN
指标 方法 a b c d
识别数
量/个		 Faster R-CNN 3 2 2 1
YOLOv3 5 2 2 1
Mask R-CNN 5 2 2 1
平均识别准确度/% Faster R-CNN 60 99.5 100 100
YOLOv3 94.2 99.5 99 100
Mask R-CNN 98.3 99.9 100 100
Tab.2  Sample graph statistics
方法 mAP/% 平均运行时间/s
Faster R-CNN 91.66 2.9
YOLOv3 97.84 0.3
Mask R-CNN 98.33 3.8
Tab.3  Test results of three methods
Tab.4  Mask R-CNN test results
[1] Chira I M, Chibulcutean A, Danescu R G. Real-time detection of road markings for driving assistance applications[C]// International Conference on Computer Engineering and Systems. IEEE, 2010:158-163.
[2] Schreiber M, Poggenhans F, Stiller C. Detecting symbols on road surface for mapping and localization using OCR[C]// IEEE International Conference on Intelligent Transportation Systems. IEEE, 2014:597-602.
[3] Wu T, Ranganathan A. A practical system for road marking detection and recognition[C]// Intelligent Vehicles Symposium. IEEE, 2012,7(2272):25-30.
[4] Maier G, Pangerl S, Schindler A. Real-time detection and classification of arrow markings using curve-based prototype fitting[C]// Intelligent Vehicles Symposium. IEEE, 2011:442-447.
[5] Ding D, Yoo J, Jekyo J, et al. Efficient road-sign detection based on machine learning[J]. Bulletin of Networking Computing Systems and Software, 2015,4(1):15-17.
[6] 胡忠闯. 基于深度学习的车道线检测算法研究[D]. 杭州:浙江大学, 2018.
[6] Hu Z C. Deep learning based lane markings detection algorithm[D]. Hangzhou:Zhejiang University, 2018.
[7] 任洪梅. 基于深度学习的路面交通标志识别[J]. 信息通信, 2017(4):26-27.
[7] Ren H M. Road traffic sign recognition based on deep learning[J]. Information and Communications, 2017(4):26-27.
[8] 张永涛. 基于机器视觉的路面交通标志识别的应用研究[D]. 广州:广东工业大学, 2015.
[8] Zhang Y T. Research on the application of pavement traffic signs recognition based on machine vision[D]. Guangzhou:Guangdong University of Technology, 2015.
[9] Danescu R, Nedevschi S. Detection and classification of painted road objects for intersection assistance applications[C]// International IEEE Conference on Intelligent Transportation Systems. IEEE, 2010:433-438.
[10] Wang N, Liu W, Zhang C M, et al. The detection and recognition of arrow markings recognition based on monocular vision[C]// IEEE International Symposium on Industrial Electronics. IEEE, 2009:4380-4386.
[11] Liu W, Lyu J, Yu B, et al. Multi-type road marking recognition using Adaboost detection and extreme learning machine classification[C]// Intelligent Vehicles Symposium (IV). IEEE, 2015:41-46.
[12] 李强. 基于多层次融合与卷积神经网络的道路交通标线检测与识别[D]. 西安:长安大学, 2018.
[12] Li Q. Road traffic marking and recognition based on multi-level fusion and convolutional neural networks[D]. Xi’an:Chang’an University, 2018.
[13] Husan V, Hyung H, Jin K, et al. Recognition of damaged arrow-road markings by visible light camera sensor based on convolutional neural network[J]. Sensors, 2016,16(12):2160.
[14] Yang T, Long X, Sangaiah A K, et al. Deep detection network for real-life traffic sign in vehicular networks[J]. Computer Networks, 2018,136:95-104.
[15] 谢锦, 蔡自兴, 邓海涛, 等. 基于图像不变特征深度学习的交通标志分类[J]. 计算机辅助设计与图形学学报, 2017,29(4):632-640.
[15] Xie J, Cai Z X, Deng H T, et al. Traffic sign classification based on deep learning of image invariant feature[J]. Journal of Computer-Aided Design and Computer Graphics, 2017,29(4):632-640.
[16] Balducci F, Impedovo D, Pirlo G. Detection and validation of tow-away road sign licenses through deep learning methods[J]. Sensors, 2018,18(12):4147.
[17] Filatov D M, Ignatiev K V, Serykh E V. Neural network system of traffic signs recognition[C]// IEEE International Conference on Soft Computing and Measurements (SCM). IEEE, 2017:1488-1491.
[18] 杨梦梦. 基于Faster R-CNN的交通标志检测方法[D]. 武汉:湖北工业大学, 2019.
[18] Yang M M. Method for traffic sign detection based on Faster R-CNN[D]. Wuhan:Hubei University of Technology, 2019.
[19] 何洪亮. 基于卷积神经网络的交通标志检测研究[D]. 南京:南京理工大学, 2018.
[19] He H L. Traffic sign detection based on convolutional neural network[D]. Nanjing:Nanjing University of Science and Technology, 2018.
[20] Zhu Y, Zhang C, Zhou D, et al. Traffic sign detection and recognition using fully convolutional network guided proposals[J]. Neurocomputing, 2016,214:758-766.
[21] 伍锡如, 雪刚刚. 基于图像聚类的交通标志CNN快速识别算法[J]. 智能系统学报, 2019,14(4):670-678.
[21] Wu X R, Xue G G. CNN-based image clustering algorithm for fast recognition of traffic signs[J]. CAAI Transactions on Intelligent Systems, 2019,14(4):670-678.
[22] 卢飞宇. 基于改进的卷积神经网络的交通标志检测与识别算法[J]. 工业控制计算机, 2019,32(6):54-56.
[22] Lu F Y. Traffic sign detection and recognition algorithm based on improved multi-task convolutional neural network[J]. Industrial Control Computer, 2019,32(6):54-56.
[23] Kryvinska N, Poniszewska-Maranda A, Greguš M. An approach towards service system building for road traffic signs detection and recognition[J]. Procedia Computer Science, 2018,141:64-71.
[24] He K M, Georgia G, Piotr D, et al. Mask R-CNN[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision, 2017:2961-2969.
[25] Ren S, He K M, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,39(6):1137-1149.
pmid: 27295650 url: https://www.ncbi.nlm.nih.gov/pubmed/27295650
[26] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2015:3431-3440.
[27] Lin T Y, Piotr D, Ross G, et al. Feature pyramid networks for object detection[C]// Proceedings of the IEEE Computer Society Conference on Computer Vision. IEEE, 2017:2117-2125.
[28] 石杰, 周亚丽, 张奇志. 基于改进Mask R-CNN和Kinect的服务机器人物品识别系统[J]. 仪器仪表学报, 2019,40(4):216-228.
[28] Shi J, Zhou Y L, Zhang Q Z. Service robot item recognition system based on improved Mask R-CNN and Kinect[J]. Chinese Journal of Scientific Instrument, 2019,40(4):216-228.
[29] He K M, Zhang X Y, Ren S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[C]// European Conference on Computer Vision.Springer, 2014:346-361.
[30] Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL]. (2018-04-08)[2020-01-07]. https://arxiv.org/abs/1804.02767.
url: https://arxiv.org/abs/1804.02767
[1] LI Penglong, DING Yi, HU Yan, LUO Ding, DUAN Songjiang, SHU Wenqiang. A method for rapid UAV images mosaicking based on GPU parallel computing[J]. REMOTE SENSING FOR LAND & RESOURCES, 2017, 29(4): 57-63.
[2] TANG Min, LI Yongshu, LI Xin, LIU Bo. Local enhancement method and its applications to UAV image matching[J]. REMOTE SENSING FOR LAND & RESOURCES, 2013, 25(4): 53-57.
[3] LU Heng, LI Yong-shu, HE Jing. A Study of the Method of Automatic Extraction of Image Points for UAV Imagery with Large Overlap[J]. REMOTE SENSING FOR LAND & RESOURCES, 2011, 23(4): 69-73.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
京ICP备05055290号-2
Copyright © 2017 Remote Sensing for Natural Resources
Support by Beijing Magtech