Please wait a minute...
 
自然资源遥感  2025, Vol. 37 Issue (6): 55-63    DOI: 10.6046/zrzyyg.2022422
  地球数据共享和知识服务 本期目录 | 过刊浏览 | 高级检索 |
北极海冰遥感观测和数值模拟数据的在线协同分析方法研究
刘昱甫1(), 徐灏1, 白玉琪1,2()
1.清华大学地球系统科学系,东亚迁徙鸟类与栖息地生态学教育部野外科学观测研究站,清华大学全球变化研究院,北京 100084
2.清华大学中国城市研究院,北京 100084
A novel method for the online collaborative analysis of Arctic sea ice data from remote sensing observations and numerical simulations
LIU Yufu1(), XU Hao1, BAI Yuqi1,2()
1. Department of Earth System Science, Ministry of Education Ecological Field Station for East Asian Migratory Birds and Their Habitatses, Institute for Global Change Studies, Tsinghua University, Beijing 100084, China
2. Tsinghua Urban Institute, Tsinghua University, Beijing 100084, China
全文: PDF(3742 KB)   HTML  
输出: BibTeX | EndNote (RIS)      
摘要 

北极海冰的快速变化对全球气候系统与极地生态环境产生深远影响。当前海冰研究高度依赖多源观测与数值模拟数据,然而传统“先下载、后分析”的数据处理模式在海量分布式数据场景下面临存储、计算与网络传输的严峻挑战。该文提出了一种基于云原生架构的在线协同分析框架,通过“计算近数据”的理念实现分布式环境数据的高效查询与跨节点协同处理。以第六阶段国际耦合模式比较计划产生的海冰模拟数据为例,系统分析了传统数据使用模式的瓶颈,设计了包含数据索引、任务管理、数据分析和Web界面的完整系统架构,并在清华大学、中国气象局和中国科学院等多机构成功部署。性能测试表明,该框架在处理分布式地学数据分析任务时,平均效率较传统模式提升13倍,且具备良好的亚线性扩展特性。研究结果为分布式地球科学数据的协同分析提供了可行方案,在线协同分析框架可为推动极地科学数据的开放协作提供有效的技术路径。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
刘昱甫
徐灏
白玉琪
关键词 北极海冰CMIP气候模式数据协同分析    
Abstract

The Arctic region is one of the most sensitive regions to global climate change in terms of response and feedback. Sea ice in the Arctic region affects the Arctic environment, ecosystems, and climate while also exerting profound influences on global ocean circulation, climate, and biodiversity. Hence, gaining a deep understanding of sea ice is critical for understanding the operational mechanisms of the Earth system, predicting climate change trends, conserving ecosystems, and advancing sustainable development. Through remote sensing observations and numerical simulations, substantial scientific data related to the historical distribution and future changes of Arctic sea ice have been acquired. These data are currently stored in large remote sensing science data centers and multiple Earth system simulation data centers involved in the Coupled Model Intercomparison Project (CMIP). However, a thorough comparative analysis of these distributed scientific data is challenged by the downloading of mass data. Based on the CMIP scientific data, this study demonstrated the difficulties encountered in data downloading. Accordingly, this study proposed a novel method and corresponding software solution for online collaborative analysis. Focusing on the sea ice data from remote sensing observations and numerical simulations, this study expounded the deployment and operation of the proposed method in multiple institutions. The proposed method can enrich the technical system for the findability, accessibility, interoperability, and reusability of the scientific data of sea ice. The demonstrated online collaborative analysis system can significantly enhance the analysis and utilization efficiency of sea ice data.

Key wordsArctic sea ice    CMIP    climate model data    collaborative analysis
收稿日期: 2022-10-28      出版日期: 2025-12-31
ZTFLH:  TP79  
基金资助:国家重点研发计划项目“面向开放科学的国际地球观测系统互操作体系研究与示范”(2019YFE0126400)
通讯作者: 白玉琪(1976-),男,博士,教授,主要从事地球和空间信息科学研究。Email: yuqibai@tsinghua.edu.cn
作者简介: 刘昱甫(1996-),男,博士研究生,从事地球科学数据基础设施研究。Email: liuyufu18@mails.tsinghua.edu.cn
引用本文:   
刘昱甫, 徐灏, 白玉琪. 北极海冰遥感观测和数值模拟数据的在线协同分析方法研究[J]. 自然资源遥感, 2025, 37(6): 55-63.
LIU Yufu, XU Hao, BAI Yuqi. A novel method for the online collaborative analysis of Arctic sea ice data from remote sensing observations and numerical simulations. Remote Sensing for Natural Resources, 2025, 37(6): 55-63.
链接本文:  
https://www.gtzyyg.com/CN/10.6046/zrzyyg.2022422      或      https://www.gtzyyg.com/CN/Y2025/V37/I6/55
Fig.1  CMIP的数据流图
Fig.2  CAFE中心和子节点示意图
接口路径 接口名称 功能备注
/modelfile/query/filter 获取查询条件初始候选值 返回数据模式各字段唯一值,供前端筛选界面使用
/modelfile/query 按条件筛选数据 根据选定字段值查询并返回匹配数据集,支持分页
/task/submit 提交分析任务 向分析后端提交含参数任务,返回提交状态与任务ID
/task/query 查询分析任务执行状态 返回任务执行进度、结果路径或失败原因等信息
Tab.1  CAFE任务相关接口
Fig.3  CAFE任务运行流程示意图
Fig.4  CAFE用户界面分析功能示意图
Fig.5  清华CAFE节点示意图
系统子域 MIP阶段 项目活动 研究机构 模型 变量
seaice CMIP6 CMIP CAMS CAMS-CSM1-0 siconc
seaice CMIP6 CMIP CCCma CanESM5 siconc
seaice CMIP6 CMIP CNRM-CERFACS CNRM-CM6-1-HR siconc
seaice CMIP6 CMIP CNRM-CERFACS CNRM-CM6-1 siconc
seaice CMIP6 CMIP CNRM-CERFACS CNRM-ESM2-1 siconc
seaice CMIP6 CMIP EC-Earth-Consortium EC-Earth3-Veg siconc
seaice CMIP6 CMIP EC-Earth-Consortium EC-Earth3 siconc
seaice CMIP6 CMIP IPSL IPSL-CM6A-LR siconc
seaice CMIP6 CMIP MIROC MIROC-ES2L siconc
seaice CMIP6 CMIP MIROC MIROC6 siconc
seaice CMIP6 CMIP MOHC HadGEM3-GC31-LL siconc
seaice CMIP6 CMIP MOHC UKESM1-0-LL siconc
seaice CMIP6 CMIP MPI-M MPI-ESM1-2-HR siconc
seaice CMIP6 CMIP MRI MRI-ESM2-0 siconc
seaice CMIP6 CMIP NCAR CESM2-WACCM siconc
seaice CMIP6 CMIP NCAR CESM2 siconc
seaice CMIP6 CMIP NCC NorCPM1 siconc
seaice CMIP6 CMIP NCC NorESM2-LM siconc
seaice CMIP6 CMIP NOAA-GFDL GFDL-CM4 siconc
seaice CMIP6 CMIP NUIST NESM3 siconc
seaice CMIP6 CMIP NCAR CESM2-FV2 siconc
seaice CMIP6 CMIP CAS FGOALS-f3-L areacello
seaice CMIP6 CMIP CNRM-CERFACS CNRM-CM6-1-HR areacello
seaice CMIP6 CMIP CNRM-CERFACS CNRM-CM6-1 areacello
seaice CMIP6 CMIP EC-Earth-Consortium EC-Earth3-Veg areacello
seaice CMIP6 CMIP EC-Earth-Consortium EC-Earth3 areacello
seaice CMIP6 CMIP MPI-M MPI-ESM1-2-HR areacello
seaice CMIP6 CMIP NCAR CESM2-FV2 areacello
seaice CMIP6 CMIP NCAR CESM2-WACCM-FV2 areacello
seaice CMIP6 CMIP NCC NorCPM1 areacello
seaice CMIP6 CMIP NCC NorESM2-LM areacello
seaice CMIP6 CMIP UA MCM-UA-1-0 areacello
seaice CMIP6 ScenarioMIP CAMS CAMS-CSM1-0 siconc
seaice CMIP6 ScenarioMIP CAS FGOALS-f3-L siconc
seaice CMIP6 ScenarioMIP CAS FGOALS-g3 siconc
seaice CMIP6 ScenarioMIP CCCma CanESM5 siconc
seaice CMIP6 ScenarioMIP CNRM-CERFACS CNRM-CM6-1 siconc
seaice CMIP6 ScenarioMIP CNRM-CERFACS CNRM-ESM2-1 siconc
seaice CMIP6 ScenarioMIP DKRZ MPI-ESM1-2-HR siconc
seaice CMIP6 ScenarioMIP EC-Earth-Consortium EC-Earth3-Veg siconc
seaice CMIP6 ScenarioMIP EC-Earth-Consortium EC-Earth3 siconc
seaice CMIP6 ScenarioMIP IPSL IPSL-CM6A-LR siconc
seaice CMIP6 ScenarioMIP MIROC MIROC-ES2L siconc
seaice CMIP6 ScenarioMIP MIROC MIROC6 siconc
seaice CMIP6 ScenarioMIP MOHC HadGEM3-GC31-LL siconc
seaice CMIP6 ScenarioMIP MOHC UKESM1-0-LL siconc
seaice CMIP6 ScenarioMIP MRI MRI-ESM2-0 siconc
seaice CMIP6 ScenarioMIP NCAR CESM2-WACCM siconc
seaice CMIP6 ScenarioMIP NCAR CESM2 siconc
seaice CMIP6 ScenarioMIP NOAA-GFDL GFDL-CM4 siconc
seaice CMIP6 ScenarioMIP NOAA-GFDL GFDL-ESM4 siconc
seaice CMIP6 ScenarioMIP NUIST NESM3 siconc
Tab.2  清华节点支持的CMIP6数据
节点 CPU 内存(RAM)/GB 存储数据量/GB
Node 1 32核Intel E5-2650 @2.00 GHz 192 355
Node 2 32核Intel E5-2650 @2.00 GHz 32 221
Node 3 32核Intel E5-2650 @2.00 GHz 64 534
Node 4 8核Intel E3-1230 @3.30 GHz 32 135
Tab.3  性能测试实验节点硬件配置信息
数据集ID 模式名称 数据源ESGF节点 数据
大小/MB
下载
耗时/s
平均下
载速度/(MB·s-1)
托管
节点
D1 inmcm4 aims3.llnl.gov 879 126 6.97 Node 1
D2 GFDL-CM3 esgdata.gfdl.noaa.gov 617 471 1.31 Node 2
D3 MIROC-ESM aims3.llnl.gov 351 61 5.76 Node 3
D4 HadGEM2-ES esgf-data1.ceda.ac.uk 520 600 0.87 Node 4
Tab.4  性能对比测试实验数据集信息
用例ID 分析数据集 协同模式 传统模式 总耗时比
节点计
算耗时/s
客户端
总耗时/s
下载耗时/s 执行耗
时/s
总耗
时/s
U1 D1 45 54 126 56 182 1∶3.37
U2 D2 61 68 471 59 530 1∶7.79
U3 D3 37 44 61 41 102 1∶2.32
U4 D4 33 34 600 50 650 1∶19.1
U5 D1&D2 46/57 65 597 86 683 1∶10.5
U6 D1&D3 45/37 52 187 78 265 1∶5.10
U7 D1&D4 45/35 51 726 76 802 1∶15.7
U8 D2&D3 59/38 67 532 85 617 1∶9.21
U9 D2&D4 56/33 64 1 071 93 1 164 1∶18.2
U10 D3&D4 37/31 43 661 67 728 1∶16.9
U11 D1&D2&D3 46/59/37 66 658 89 747 1∶11.3
U12 D1&D2&D4 45/59/30 66 1 197 95 1 292 1∶19.6
U13 D2&D3&D4 61/37/35 68 1 132 93 1 225 1∶18.0
U14 D1&D3&D4 47/38/33 51 787 79 866 1∶17.0
U15 D1&D2&D3&D4 46/63/37/31 70 1 258 102 1 360 1∶19.4
平均值 57.5 670.9 76.6 747.5 1∶13.0
Tab.5  不同测试用例下2种方案的性能结果比较
Fig.6  多节点协同分析方案的可扩展性评估结果图示
[1] 唐述林, 秦大河, 任贾文, 等. 极地海冰的研究及其在气候变化中的作用[J]. 冰川冻土, 2006, 28(1):91-100.
Tang S L, Qin D H, Ren J W, et al. The studies of polar sea ice and their contribution to climate change researches[J]. Journal of Glaciology and Geocryology, 2006, 28(1):91-100.
[2] Riihelä A, Bright R M, Anttila K. Recent strengthening of snow and ice albedo feedback driven by Antarctic sea-ice loss[J]. Nature Geoscience, 2021, 14(11):832-836.
doi: 10.1038/s41561-021-00841-x
[3] Di Biagio C, Pelon J, Blanchard Y, et al. Toward a better surface radiation budget analysis over sea ice in the high Arctic Ocean:A comparative study between satellite,reanalysis,and local-scale observations[J]. Journal of Geophysical Research:Atmospheres, 2021, 126(4):e2020JD032555.
[4] Jäkel E, Sperzel T R, Wendisch M, et al. What determines the Arctic solar radiation energy budget at the surface most strongly:Clouds,surface albedo,or the solar zenith angle?[J]. Journal of the European Meteorological Society, 2025,3:100016.
[5] Zhu J L, Liu Y L, Wang X Y, et al. Optical properties and surface energy flux of spring fast ice in the Arctic[J]. Acta Oceanologica Sinica, 2021, 40(10):84-96.
[6] Doney S C, Ruckelshaus M, Emmett Duffy J, et al. Climate change impacts on marine ecosystems[J]. Annual Review of Marine Science, 2012,4:11-37.
[7] Blanchet M A, Primicerio R, Frainer A, et al. The role of marine mammals in the Barents Sea foodweb[J]. ICES Journal of Marine Science, 2019, 76(Supplement_1):i37-i53.
[8] Pagano A M, Durner G M, Atwood T C, et al. Effects of sea ice decline and summer land use on polar bear home range size in the Beaufort Sea[J]. Ecosphere, 2021, 12(10):e03768.
doi: 10.1002/ecs2.v12.10
[9] Herman-Mercer N M, Laituri M, Massey M, et al. Vulnerability of subsistence systems due to social and environmental change[J]. Arctic, 2019, 72(3):258-272.
doi: 10.14430/arctic68867
[10] Rantanen M, Karpechko A Y, Lipponen A, et al. The Arctic has warmed nearly four times faster than the globe since 1979[J]. Communications Earth & Environment, 2022,3:168.
[11] Serreze M C, Barry R G. Processes and impacts of Arctic amplification:A research synthesis[J]. Global and Planetary Change, 2011, 77(1/2):85-96.
doi: 10.1016/j.gloplacha.2011.03.004
[12] Screen J A, Simmonds I. The central role of diminishing sea ice in recent Arctic temperature amplification[J]. Nature, 2010, 464(7293):1334-1337.
doi: 10.1038/nature09051
[13] Stroeve J C, Serreze M C, Holland M M, et al. The Arctic’s rapidly shrinking sea ice cover:A research synthesis[J]. Climatic Change, 2012, 110(3):1005-1027.
doi: 10.1007/s10584-011-0101-1
[14] Comiso J C, Nishio F. Trends in the sea ice cover using enhanced and compatible AMSR-E,SSM/I,and SMMR data[J]. Journal of Geophysical Research:Oceans, 2008, 113(C2):2007JC004257.
[15] 季青, 庞小平, 许苏清, 等. 极地海冰厚度探测方法及其应用研究综述[J]. 极地研究, 2016, 28(4):431-441.
Ji Q, Pang X P, Xu S Q, et al. Review of technology and application research on polar sea ice thickness detection[J]. Chinese Journal of Polar Research, 2016, 28(4):431-441.
doi: 10.13679/j.jdyj.2016.4.431
[16] 周天军, 邹立维, 陈晓龙. 第六次国际耦合模式比较计划(CMIP6)评述[J]. 气候变化研究进展, 2019, 15(5):445-456.
Zhou T J, Zou L W, Chen X L. Commentary on the coupled model intercomparison project phase 6(CMIP6)[J]. Climate Change Research, 2019, 15(5):445-456.
[17] Cinquini L, Crichton D, Mattmann C, et al. The Earth System Grid Federation:An open infrastructure for access to distributed geospatial data[C]//2012 IEEE 8th International Conference on E-Science.October 8-12,2012, Chicago,IL,USA.IEEE, 2013:1-10.
[18] Liu Y, Wang M, Chen L, et al. A collaborative analysis framework for environmental data (CAFE)[J]. Environmental Modelling & Software, 2018,99:113-126.
[1] 刘昱甫, 白玉琪. 国际耦合模式比较计划地球模拟数据全球共享体系分析[J]. 自然资源遥感, 2025, 37(6): 10-21.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
京ICP备05055290号-2
版权所有 © 2015 《自然资源遥感》编辑部
地址:北京学院路31号中国国土资源航空物探遥感中心 邮编:100083
电话:010-62060291/62060292 E-mail:zrzyyg@163.com
本系统由北京玛格泰克科技发展有限公司设计开发