Abstract:
Multi-class semantic change detection (SCD) from remote sensing images represents a highly challenging task as it utilizes only pixel-level change labels rather than bi-temporal semantic segmentation labels. In recent years, vision foundation models (VFMs) pretrained on massive natural images have quickly adapted to downstream tasks due to their parameter-efficient fine-tuning, demonstrating high generalization ability. However, their potential and applicability in SCD from remote sensing images remain poorly understood. This study systematically compared parameter-efficient SCD methods built on pretrained weights of models including the Segment Anything Model (SAM) and self-distillation with no labels (DINO). Furthermore, the differences in accuracy and performance among the methods were assessed. First, multi-source remote sensing SCD datasets, such as Gaza-Change and Landsat-SCD, were selected. Then, a unified encoder-decoder architecture was designed, in which the image encoders and pretrained weights of SAM and DINO were loaded separately under strict control of variables. This architecture served as the feature-extraction backbone. Finally, a comprehensive comparative analysis was conducted based on multiple quantitative metrics, including overall accuracy (OA), mean Intersection over Union (IoU), F-score for SCD (Fscd), and separated Kappa coefficient (SeK), as well as the visual quality of change maps. Experimental results demonstrate that owing to its strong semantic representation capability, the DINO model for pretraining yielded higher overall accuracy in the detection of SCD tasks. The advantages of this DINO model were particularly prominent in distinguishing semantic changes such as building and road changes. In contrast, the SAM model performed well in edge detail preservation and fine-scale segmentation. However, it showed limited zero-shot generalization ability for the concept of change, leading to limited performance in direct applications. After an asymmetric dual-encoder architecture was applied, the DINO and ResNet combined model achieved effective improvement in OA and visual performance, suggesting that features of different encoders are mutually complementary. This study provides both theoretical guidance and a practical reference for VFM selection and pretraining in SCD from remote sensing images.