高级检索

    预训练视觉基础模型在多类别遥感语义变化检测中的对比研究

    A comparative study of the application of pretrained vision foundation models to multi-class semantic change detection of remote sensing images

    • 摘要: 多类别遥感语义变化检测仅使用遥感图像中的变化像素标签进行语义变化检测,不依赖双时相语义分割标签,任务挑战大。近年来,基于海量自然图像预训练的视觉基础模型凭借少量参数微调方式快速适应下游任务,展现出强大的泛化能力,但其在遥感语义变化检测任务中的潜力和适应性尚不明确。该文旨在系统性地对比基于分割一切模型(segment anything model,SAM)和自监督视觉模型(self-distillation with no labels,DINO)等预训练权重的参数高效语义变化检测方法,评估其精度与性能差异。首先,选取Gaza-Change,Landsat-SCD等异源遥感语义变化检测数据集; 其次,设计统一的编码器-解码器框架,按照控制变量方式分别加载SAM, DINO的图像编码器和预训练权重作为特征提取主干网络; 最后,从总体精度(overall accuracy,OA)、平均交并比(mean intersection over union, mIoU)、语义变化检测F分数(F-score for semantic change detection, Fscd)以及分离Kappa系数(separated Kappa, SeK)等多个定量指标,以及变化图视觉质量进行综合对比分析。实验结果表明,DINO系列预训练模型由于其强大的语义特征表示能力,在变化检测任务上总体精度更高,尤其在区分“建筑变化”与“道路变化”等语义性变化时优势明显; SAM模型在边缘细节保持和分割精细化方面表现出色,但其对“变化”概念的零样本泛化能力有限,直接应用效果不佳。在应用非对称双编码器结构后,DINO系列和ResNet的组合模型精度和可视效果得到有效提升,说明不同编码器特征能够有效互补。该研究为遥感语义变化领域选择和预训练视觉基础模型提供了理论依据和实践参考。

       

      Abstract: Multi-class semantic change detection (SCD) from remote sensing images represents a highly challenging task as it utilizes only pixel-level change labels rather than bi-temporal semantic segmentation labels. In recent years, vision foundation models (VFMs) pretrained on massive natural images have quickly adapted to downstream tasks due to their parameter-efficient fine-tuning, demonstrating high generalization ability. However, their potential and applicability in SCD from remote sensing images remain poorly understood. This study systematically compared parameter-efficient SCD methods built on pretrained weights of models including the Segment Anything Model (SAM) and self-distillation with no labels (DINO). Furthermore, the differences in accuracy and performance among the methods were assessed. First, multi-source remote sensing SCD datasets, such as Gaza-Change and Landsat-SCD, were selected. Then, a unified encoder-decoder architecture was designed, in which the image encoders and pretrained weights of SAM and DINO were loaded separately under strict control of variables. This architecture served as the feature-extraction backbone. Finally, a comprehensive comparative analysis was conducted based on multiple quantitative metrics, including overall accuracy (OA), mean Intersection over Union (IoU), F-score for SCD (Fscd), and separated Kappa coefficient (SeK), as well as the visual quality of change maps. Experimental results demonstrate that owing to its strong semantic representation capability, the DINO model for pretraining yielded higher overall accuracy in the detection of SCD tasks. The advantages of this DINO model were particularly prominent in distinguishing semantic changes such as building and road changes. In contrast, the SAM model performed well in edge detail preservation and fine-scale segmentation. However, it showed limited zero-shot generalization ability for the concept of change, leading to limited performance in direct applications. After an asymmetric dual-encoder architecture was applied, the DINO and ResNet combined model achieved effective improvement in OA and visual performance, suggesting that features of different encoders are mutually complementary. This study provides both theoretical guidance and a practical reference for VFM selection and pretraining in SCD from remote sensing images.

       

    /

    返回文章
    返回