Abstract: Vehicle detection is a hot research topic in the fields of computer vision, photogrammetry, and remote sensing. With the continuous development of deep learning technology, vehicle detection based on remote sensing images has been applied in fields such as smart city construction and intelligent transportation. This study systematically summarized existent vehicle detection algorithms based on remote sensing images and deep learning models and highlighted the classification, analysis, and comparison of one-stage and two-stage vehicle detection algorithms. Moreover, this study summarized the key technologies of vehicle detection in large-scale and complex backgrounds and analyzed the advantages and disadvantages of mainstream deep learning models of vehicle detection based on remote sensing images. Experiments were conducted to evaluate the YOLOv5, Faster-RCNN, FCOS, and SSD algorithms using DOTA and DIOR datasets. The vehicle detection precision based on the DOTA dataset was 0.695, 0.410, 0.370, and 0.251, respectively and that based on the DIOR dataset was 0.566, 0.243, 0.231, and 0.154, respectively. The experimental results show that the small target scale is still the main factor restricting the vehicle detection performance based on remote sensing images and that the application of deep learning models to the detection of small targets is to be further improved. Finally, based on public datasets and the analysis of existing algorithms, this study proposed the solution and development trend of vehicle detection based on remote sensing images in large-scale and complex backgrounds.