With continuously accelerated urbanization, urban expansion-induced farmland occupation and rural hollowing have gradually aggravated arable land abandonment, posing challenges to China’s food security. Hence, accurately determining the distribution of abandoned arable land is critical to arable land protection and food security. This study investigated a county in the major crop production area in a hilly region in southern China. Based on the phenological characteristics of local rice planting, this study adopted six phases of satellite remote sensing images and aerial images acquired in 2020 and 2021 were used as the data source and selected paddy fields determined based on the Third National Land Survey and field ridges in 1∶2 000 digital line graphics (DLGs) as the minimum information extraction unit. Then, the rice planting patches were extracted through time-series normalized difference vegetation index (NDVI) analysis and the improved SVM method. Suspected abandoned areas were screened out through the difference calculation for two consecutive years and then further identified in the unmanned aerial vehicle (UAV)-based sampling aerial survey. Consequently, the abandoned arable land areas were determined, with monitoring accuracy exceeding 85%, as revealed by on-site verification. The results of this study show that the space-air-ground integrated remote sensing monitoring method can provide scientific and effective data support for agricultural management departments to control and manage cultivated land abandonment.