Abstract: The scene recognition of shipbuilding enterprises is of practical significance for the restoration of the coastal ecological environment, the protection of water environment, and the promotion of the coordinated development of shipbuilding enterprises. However, it is difficult to realize the automatic recognition of shipbuilding enterprises from satellite remote sensing images based on traditional medium- and low-level features. Therefore, this paper proposes a multi-model multi-scale scene recognition method of shipbuilding enterprises based on a convolutional neural network with spatial constraints and the steps are as follows. Firstly, train multiple convolutional neural network models using the samples of global-scale shipbuilding enterprise scenes and local-scale docks (slipways), workshops, and ships individually, and conduct multi-model multi-scale detection. Then, locate local-scale objects at a pixel level and calculate the spatial distance of the objects. Finally, conduct comprehensive judgment and extraction of the shipbuilding enterprise scenes according to the multi-scale detection results, the combination method of object tags, and the spatial distance of objects. The method was applied to five typical shipbuilding intensive areas in Jiangsu Province, China, the surrounding areas of Nagasaki and Ehime prefectures, Japan, and Mokpo and Geoje cities, South Korea. As a result, the overall recognition accuracy and recall rate were 87% and 85%, respectively in Jiangsu Province, were 91% and 87%, respectively in the study area in Japanese, and were 85% and 92%, respectively in the study area in South Korean. The experimental results show that this method can realize the effective recognition of the complex scenes of shipbuilding enterprises based on remote sensing images.