Multi-model and multi-scale scene recognition of shipbuilding enterprises based on convolutional neural network with spatial constraints

doi:10.6046/zrzyyg.2021020

Abstract
Figures/Tables
References
Related Articles
Metrics

Download: PDF(9885 KB) HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks

Abstract

The scene recognition of shipbuilding enterprises is of practical significance for the restoration of the coastal ecological environment, the protection of water environment, and the promotion of the coordinated development of shipbuilding enterprises. However, it is difficult to realize the automatic recognition of shipbuilding enterprises from satellite remote sensing images based on traditional medium- and low-level features. Therefore, this paper proposes a multi-model multi-scale scene recognition method of shipbuilding enterprises based on a convolutional neural network with spatial constraints and the steps are as follows. Firstly, train multiple convolutional neural network models using the samples of global-scale shipbuilding enterprise scenes and local-scale docks (slipways), workshops, and ships individually, and conduct multi-model multi-scale detection. Then, locate local-scale objects at a pixel level and calculate the spatial distance of the objects. Finally, conduct comprehensive judgment and extraction of the shipbuilding enterprise scenes according to the multi-scale detection results, the combination method of object tags, and the spatial distance of objects. The method was applied to five typical shipbuilding intensive areas in Jiangsu Province, China, the surrounding areas of Nagasaki and Ehime prefectures, Japan, and Mokpo and Geoje cities, South Korea. As a result, the overall recognition accuracy and recall rate were 87% and 85%, respectively in Jiangsu Province, were 91% and 87%, respectively in the study area in Japanese, and were 85% and 92%, respectively in the study area in South Korean. The experimental results show that this method can realize the effective recognition of the complex scenes of shipbuilding enterprises based on remote sensing images.

Keywords satellite remote sensing shipbuilding enterprises convolutional neural networks multi-scale spatial distance constraint

ZTFLH:

TP79

Corresponding Authors: SONG Yan E-mail: yu_xinli@cug.edu.cn;songyan@cug.edu.cn

Issue Date: 23 December 2021

	Service

	E-mail this article
	E-mail Alert
	RSS
	Articles by authors

	Xinli YU
	Yan SONG
	Miao YANG
	Lei HUANG
	Yanjie ZHANG

Cite this article:

Xinli YU,Yan SONG,Miao YANG, et al. Multi-model and multi-scale scene recognition of shipbuilding enterprises based on convolutional neural network with spatial constraints[J]. Remote Sensing for Natural Resources, 2021, 33(4): 72-81.

URL:

https://www.gtzyyg.com/EN/10.6046/zrzyyg.2021020 OR https://www.gtzyyg.com/EN/Y2021/V33/I4/72

Fig.1 Internal structure diagram of the shipbuilding enterprise

Tab.1 Interpretation marks within the shipbuilding enterprise

Fig.2 Residual learning unit

Fig.3 Multi-model detection process

Fig.4 Multi-model and multi-scale shipbuilding enterprise recognition process

Fig.5 Local object spatial relationship constraints

Fig.6 Scene recognition process of shipbuilding enterprise

Tab.2 Number of samples

Tab.3 Network training results(%)

Fig.7 Display of the results of the study area in Jiangsu Province

Fig.8 Display of results of the study area in Japan

Fig.9 Display of results of the study area in Korea

Tab.4 Recognition results of shipbuilding enterprises in Jiangsu Province based on different basic network models(%)

Fig.10 Final recognition results of the distribution of shipbuilding enterprises in Jiangsu Province

Tab.5 Detailed results in Jiangsu Province

Fig.11 The recognition results of shipbuilding enterprises in Japan study area

Tab.6 Detailed display of recognition results of shipbuilding enterprises in Japanese research areas

Tab.7 Detailed display of recognition results of shipbuilding enterprises in Korean research areas

Fig.12 The recognition results of shipbuilding enterprises in Korea study area

[1]	杨青. 长三角船舶产业集群研究[D]. 上海:上海海事大学, 2007.
[1]	Yang Q. Research on the shipbuilding industry cluster in the Yangtze River delta[D]. Shanghai:Shanghai Maritime University, 2007.
[2]	刘辉, 史雅娟, 曾春水. 中国船舶产业空间布局与发展策略[J]. 经济地理, 2017,37(8):99-107.
[2]	Liu H, Shi Y J, Zeng C S. Spatial layout and development strategy of the ship-related industry in China[J]. Economic Geography, 2017,37(8):99-107.
[3]	Zhou Z H, Zhang M L, Huang S J, et al. Multi-instance and multi-label learning[J]. Artificial Intelligence, 2012,176(1):2291-2320. doi: 10.1016/j.artint.2011.10.002 url: https://linkinghub.elsevier.com/retrieve/pii/S0004370211001123
[4]	Li Z L, Xu K, Xie J F, et al. Deep multiple instance convolutional neural networks for learning robust scene representations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020,58(5):3685-3702. doi: 10.1109/TGRS.36 url: https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=36
[5]	Liu W H, Li Y D, Wu Q. An attribute-based high-level image representation for scene classification[J]. IEEE Access, 2019,7:4629-4640. doi: 10.1109/ACCESS.2018.2886597 url: https://ieeexplore.ieee.org/document/8574892/
[6]	Khan N, Chaudhuri U, Banerjee B, et al. Graph convolutional network for multi-label VHR remote sensing scene recognition[J]. Neurocomputing, 2019,357:36-46. doi: 10.1016/j.neucom.2019.05.024 url: https://linkinghub.elsevier.com/retrieve/pii/S0925231219306861
[7]	Chen Z M, Wei X S, Wang P, et al. Multi-Label Image Recognition with Graph Convolutional Networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE, 2019.
[8]	Clement M, Kurtz C, Wendling L. Learning spatial relations and shapes for structural object description and scene recognition[J]. Pattern Recognition, 2018,84:197-210. doi: 10.1016/j.patcog.2018.06.017 url: https://linkinghub.elsevier.com/retrieve/pii/S0031320318302292
[9]	Lazebnik S, Schmid C, Ponce J. Beyond bags of features:Spatial pyramid matching for recognizing natural scene categories[C]//Computer Vision and Pattern Recognition,2006 IEEE Computer Society Conference, 2006.
[10]	Jiang Y, Yuan J, Gang Y. Randomized spatial partition for scene recognition[M]. Berlin:Springer Berlin Heidelberg, 2012:730-743.
[11]	Weng C Q, Wang H X, Yuan J S, et al. Discovering Class-Specific Spatial Layouts for Scene Recognition[J]. IEEE Signal Processing Letters, 2017,24(8):1143-1147. doi: 10.1109/LSP.2016.2641020 url: http://ieeexplore.ieee.org/document/7786827/
[12]	He K M, Zhang X Y, Ren S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(9):1904-1916. doi: 10.1109/TPAMI.2015.2389824 url: http://ieeexplore.ieee.org/document/7005506/
[13]	Xie L, Lee F F, Liu L, et al. Scene recognition:A comprehensive survey[J]. Pattern Recognition, 2020,102:107205. doi: 10.1016/j.patcog.2020.107205 url: https://linkinghub.elsevier.com/retrieve/pii/S003132032030011X
[14]	船舶工业年鉴编委会. 中国船舶工业年鉴2018[Z].[s.l]:船舶工业年鉴编委会, 2018.
[14]	Editorial Committee of Yearbook of Shipbuilding Industry. Yearbook of China shipbuilding industry 2018[Z].[s.l]:Editorial Committee of Yearbook of Shipbuilding Industry, 2018.
[15]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas:2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2016:770-778.
[16]	颜荔. 基于卷积神经网络的遥感图像飞机目标识别研究[D]. 合肥:中国科学技术大学, 2018.
[16]	Yan L. Research on airplane target recognition in remote sensing images based on convolutional neural network[D]. Hefei:University of Science and Technology of China, 2018.
[17]	Zhou Z H. A brief introduction to weakly supervised learning[J]. National Science Review, 2018,5(1):44-53. doi: 10.1093/nsr/nwx106 url: https://academic.oup.com/nsr/article/5/1/44/4093912
[18]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM:Visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020,128(2):336-359. doi: 10.1007/s11263-019-01228-7 url: https://doi.org/10.1007/s11263-019-01228-7
[19]	Xia G S, Hu J W, Hu F, et al. AID:A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(7):3965-3981. doi: 10.1109/TGRS.2017.2685945 url: http://ieeexplore.ieee.org/document/7907303/
[20]	Cheng G, Han J W, Lu X Q. Remote sensing image scene classification:Benchmark and state of the art[J]. Proceedings of the IEEE, 2017,105(10):1865-1883. doi: 10.1109/JPROC.2017.2675998 url: http://ieeexplore.ieee.org/document/7891544/
[21]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6):84-90. doi: 10.1145/3065386 url: https://dl.acm.org/doi/10.1145/3065386
[22]	Karen S, Andrew Z. Very deep convolutional networks for large-scale image recognition[J]. arXiv, 2014.

[1]	LI Kailin, LIAO Kuo, DANG Haofei. Recent progress in chromaticity remote sensing of inland and nearshore water bodies[J]. Remote Sensing for Natural Resources, 2023, 35(1): 15-26.
[2]	SU Tengfei. A unsupervised quality valuation method for multi-scale remote sensing image segmentation based on boundary information[J]. Remote Sensing for Natural Resources, 2023, 35(1): 35-40.
[3]	SHEN Jun’ao, MA Mengting, SONG Zhiyuan, LIU Tingzhou, ZHANG Wei. Water information extraction from high-resolution remote sensing images using the deep-learning based semantic segmentation model[J]. Remote Sensing for Natural Resources, 2022, 34(4): 129-135.
[4]	WU Linlin, LI Xiaoyan, MAO Dehua, WANG Zongming. Urban land use classification based on remote sensing and multi-source geographic data[J]. Remote Sensing for Natural Resources, 2022, 34(1): 127-134.
[5]	GAO Qi, WANG Yuzhen, FENG Chunhui, MA Ziqiang, LIU Weiyang, PENG Jie, JI Yanzhen. Remote sensing inversion of desert soil moisture based on improved spectral indices[J]. Remote Sensing for Natural Resources, 2022, 34(1): 142-150.
[6]	WEN Yintang, WANG Tiezhu, WANG Shutao, Wang Guichuan, LIU Shiyu, CUI Kai. Automatic extraction of mosaic lines from high-resolution remote sensing images based on multi-scale segmentation[J]. Remote Sensing for Natural Resources, 2021, 33(4): 64-71.
[7]	JIANG Yanan, ZHANG Xin, ZHANG Chunlei, ZHONG Chengcheng, ZHAO Junfang. Classification of remote sensing images based on multi-scale feature fusion using local binary patterns[J]. Remote Sensing for Natural Resources, 2021, 33(3): 36-44.
[8]	LIU Wanjun, GAO Jiankang, QU Haicheng, JIANG Wentao. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3): 97-106.
[9]	JIANG Xiao, ZHONG Chang, LIAN Zheng, WU Liangting, SHAO Zhitao. Research progress on classification criterion of geological information products based on satellite remote sensing[J]. Remote Sensing for Natural Resources, 2021, 33(3): 279-283.
[10]	Hailing GU, Chao CHEN, Ying LU, Yanli CHU. Construction of regional economic development model based on satellite remote sensing technology[J]. Remote Sensing for Land & Resources, 2020, 32(2): 226-232.
[11]	Decai JIANG, Wenji LI, Jingmin LI, Zhaofeng BAI. Extraction of the forest fire region based on the span of ALOS PALSAR by object-oriented analysis[J]. Remote Sensing for Land & Resources, 2019, 31(4): 47-52.
[12]	Yufeng LIU, Ying PAN, Hu LI. *Study of crown information extraction of Picea schrenkiana var. tianschanicabased on high-resolution satellite remote sensing data*[J]. Remote Sensing for Land & Resources, 2019, 31(4): 112-119.
[13]	Dechao ZHAI, Yanan FAN, Yanan ZHOU. Multi-scale segmentation of satellite imagery by edge-incorporated weighted aggregation[J]. Remote Sensing for Land & Resources, 2019, 31(3): 36-42.
[14]	Ning MAO, Huiping LIU, Xiangping LIU, Yanghua ZHANG. Optimal scale selection for multi-scale segmentation based on RMNE method[J]. Remote Sensing for Land & Resources, 2019, 31(2): 10-16.
[15]	Liuqing WU, Xiangyun HU. Automatic building detection of high-resolution remote sensing images based on multi-scale and multi-feature[J]. Remote Sensing for Land & Resources, 2019, 31(1): 71-78.

Viewed

Full text

Abstract

Cited

Shared

Discussed