Review of spatiotemporal fusion model of remote sensing data
Wenquan DONG1,2(), Jihua MENG1()
1. Key Laboratory of Digital Earth Science, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101,China 2. University of Chinese Academy of Sciences, Beijing 100049, China
Taking the interaction between spatial and temporal resolution of remote sensing data into consideration, the authors hold that there is no satellite sensor that can produce images with both high spatial and temporal resolution, and spatiotemporal fusion of remote sensing data is an effective method to solve this problem. This paper introduces main research achievements of spatiotemporal fusion model obtained both in China and abroad. Based on the comparative analysis of the mainstream fusion models, these models can be divided into two categories, i.e., the transformation-based model and the pixel-reconstruction-based model. Furthermore, the authors divide the pixel-reconstruction-based model into mixed linear model and spatial and temporal adaptive reflectance model, and then introduce the basic principles, methods of these models. This paper makes a comparative analysis of the advantages and disadvantages of various aspects of the model. At last, the data, application and scale prospect of spatiotemporal fusion models are put forward.
Meng J H, Wu B F, Li Q Z , et al. Research advances and outlook of crop monitoring with remote sensing at field level[J].Remote Sensing Information, 2010(3):122-128.
[2]
Emelyanova I V, McVicar T R,Niel T G V,et al. Assessing the accuracy of blending Landsat-MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics:A framework for algorithm selection[J]. Remote Sensing of Environment, 2013,133:193-209.
doi: 10.1016/j.rse.2013.02.007
[3]
Price J C . How unique are spectral signatures?[J]. Remote Sensing of Environment, 1994,49(3):181-186.
doi: 10.1016/0034-4257(94)90013-2
[4]
Zhang W, Li A N, Jin H A , et al. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data[J]. Remote Sensing, 2013,5(10):5346-5368.
doi: 10.3390/rs5105346
Li X, Huang C L, Che T , et al. Development of a Chinese land data assimilation system:Its progress and prospects[J]. Progress in Natural Science, 2007,17(2):163-173.
[6]
Shabanov N V, Wang Y, Buermann W , et al. Effect of foliage spatial heterogeneity in the MODIS LAI and FPAR algorithm over broadleaf forests[J]. Remote Sensing of Environment, 2003,85(4):410-423.
doi: 10.1016/S0034-4257(03)00017-8
[7]
Hilker T, Wulder M A, Coops N C , et al. Generation of dense time series synthetic Landsat data through data blending with MODIS using a spatial and temporal adaptive reflectance fusion model[J]. Remote Sensing of Environment, 2009,113(9):1988-1999.
doi: 10.1016/j.rse.2009.05.011
[8]
Pohl C, Genderen J L V . Review article multisensor image fusion in remote sensing:Concepts,methods and applications[J]. International Journal of Remote Sensing, 1998,19(5):823-854.
doi: 10.1080/014311698215748
Gu X H, Han L J, Wang J H , et al. Estimation of maize planting area based on wavelet fusion of multi-resolution images[J]. Transactions of the CSAE, 2012,28(3):203-209.
[10]
Acerbi-Junior F, Clevers J G P W,Schaepman M E .The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian Savanna[J]. International Journal of Applied Earth Observation and Geoinformation, 2006,8(4):278-288.
doi: 10.1016/j.jag.2006.01.001
[11]
Wu M Q, Wang C Y.Spatial and temporal fusion of remote sensing data using wavelet transform[C]//Proceedings of 2011 International Conference on Remote Sensing,Environment and Transportation Engineering (RSETE). Nanjing:IEEE, 2011: 1581-1584.
[12]
Shevyrnogov A, Trefois P, Vysotskaya G . Multi-satellite data merge to combine NOAA AVHRR efficiency with Landsat-6 MSS spatial resolution to study vegetation dynamics[J]. Advances in Space Research, 2000,26(7):1131-1133.
doi: 10.1016/S0273-1177(99)01130-8
[13]
Hansen M C, Roy D P, Lindquist E , et al. A method for integrating MODIS and Landsat data for systematic monitoring of forest cover and change in the Congo Basin[J]. Remote Sensing of Environment, 2008,112(5):2495-2513.
doi: 10.1016/j.rse.2007.11.012
[14]
Zhukov B, Oertel D, Lanzl F , et al. Unmixing-based multisensor multiresolution image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 1999,37(3):1212-1226.
doi: 10.1109/36.763276
[15]
Maselli F . Definition of spatially variable spectral endmembers by locally calibrated multivariate regression analyses[J]. Remote Sensing of Environment, 2001,75(1):29-38.
doi: 10.1016/S0034-4257(00)00153-X
[16]
Busetto L, Meroni M, Colombo R . Combining medium and coarse spatial resolution satellite data to improve the estimation of sub-pixel NDVI time series[J]. Remote Sensing of Environment, 2008,112(1):118-131.
doi: 10.1016/j.rse.2007.04.004
[17]
Gao F, Masek J, Schwaller M , et al. On the blending of the Landsat and MODIS surface reflectance:Predicting daily Landsat surface reflectance[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006,44(8):2207-2218.
doi: 10.1109/TGRS.2006.872081
[18]
Hilker T, Wulder M A, Coops N C , et al. A new data fusion model for high spatial-and temporal-resolution mapping of forest disturbance based on Landsat and MODIS[J]. Remote Sensing of Environment, 2009,113(8):1613-1627.
doi: 10.1016/j.rse.2009.03.007
[19]
Zhu X L, Chen J, Gao F , et al. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions[J]. Remote Sensing of Environment, 2010,114(11):2610-2623.
doi: 10.1016/j.rse.2010.05.032
Meng J H, Wu B F, Du X , et al. Method to construct high spatial and temporal resolution NDVI dataset-STAVFM[J]. Journal of Remote Sensing, 2011,15(1):44-59.
[21]
Meng J H, Du X, Wu B F . Generation of high spatial and temporal resolution NDVI and its application in crop biomass estimation[J]. International Journal of Digital Earth, 2013,6(3):203-218.
doi: 10.1080/17538947.2011.623189
[22]
Amolins K, Zhang Y, Dare P . Wavelet based image fusion techniques:An introduction,review and comparison[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2007,62(4):249-263.
doi: 10.1016/j.isprsjprs.2007.05.009
[23]
Thomas C, Ranchin T, Wald L , et al. Synjournal of multispectral images to high spatial resolution:A critical review of fusion methods based on remote sensing physics[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008,46(5):1301-1312.
doi: 10.1109/TGRS.2007.912448
[24]
Malenovský Z, Bartholomeus H M , Acerbi-Junior F W,et al.Scaling dimensions in spectroscopy of soil and vegetation[J]. International Journal of Applied Earth Observation and Geoinformation, 2007,9(2):137-164.
doi: 10.1016/j.jag.2006.08.003
[25]
何馨 . 基于多源数据融合的玉米种植面积遥感提取研究[D]. 南京:南京信息工程大学, 2010.
He X . Study on Extraction of Maize Planting Area Based on Multi Source Remote Sensing Fusion Data[D]. Nanjing: Nanjing University of Information Science and Technology, 2010.
[26]
Cherchali S, Amram O, Flouzat G . Retrieval of temporal profiles of reflectances from simulated and real NOAA-AVHRR data over heterogeneous landscapes[J]. International Journal of Remote Sensing, 2000,21(4):753-775.
doi: 10.1080/014311600210551
[27]
Haertel V F, Shimabukuro Y E . Spectral linear mixing model in low spatial resolution image data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005,43(11):2555-2562.
doi: 10.1109/IGARSS.2004.1369815
[28]
Fortin J P, Bernier M, Lapointe S , et al. Estimation of Surface Variables at the Sub-Pixel Level for Use As Input to Climate and Hydrological Models[R].Québec: INRS-Eau, 1998.
[29]
Maselli F, Gilabert M A, Conese C . Integration of high and low resolution NDVI data for monitoring vegetation in Mediterranean environments[J]. Remote Sensing of Environment, 1998,63(3):208-218.
doi: 10.1016/S0034-4257(97)00131-4
[30]
Potapov P, Hansen M C, Stehman S V , et al. Combining MODIS and Landsat imagery to estimate and map boreal forest cover loss[J]. Remote Sensing of Environment, 2008,112(9):3708-3719.
doi: 10.1016/j.rse.2008.05.006
[31]
Huang B, Zhang H K . Spatio-temporal reflectance fusion via unmixing:Accounting for both phenological and land-cover changes[J]. International Journal of Remote Sensing, 2014,35(16):6213-6233.
doi: 10.1080/01431161.2014.951097
Shi Y C, Yang G J, Li X C , et al. Intercomparison of the different fusion methods for generating high spatial-temporal resolution data[J]. Journal of Infrared and Millimeter Waves, 2015,34(1):92-99.
[33]
Zurita-Milla R, Kaiser G, Clevers J G P W,et al. Downscaling time series of MERIS full resolution data to monitor vegetation seasonal dynamics[J]. Remote Sensing of Environment, 2009,113(9):1874-1885.
doi: 10.1016/j.rse.2009.04.011
Wu M Q, Wang J, Niu Z , et al. A model for spatial and temporal data fusion[J]. Journal of Infrared and Millimeter Waves, 2012,31(1):80-84.
[35]
Wu M Q, Niu Z, Wang C Y , et al. Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model[J]. Journal of Applied Remote Sensing, 2012,6(1):063507.
doi: 10.1117/1.JRS.6.063507
Wu M Q, Niu Z, Wang C Y . Mapping paddy fields by using spatial and temporal remote sensing data fusion technology[J]. Transactions of the CSAE, 2010,26(2):48-52.
Xie D F, Zhang J S, Pan Y Z , et al. Fusion of MODIS and Landsat8 images to generate high spatial-temporal resolution data for mapping autumn crop distribution[J]. Journal of Remote Sensing, 2015,19(5):791-805.
[38]
Gevaert C M, García-Haro F J. A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion[J]. Remote Sensing of Environment, 2015,156:34-44.
doi: 10.1016/j.rse.2014.09.012
[39]
Fu D J, Chen B Z, Wang J , et al. An improved image fusion approach based on enhanced spatial and temporal the adaptive reflectance fusion model[J]. Remote Sensing, 2013,5(12):6346-6360.
doi: 10.3390/rs5126346
[40]
Roy D P, Ju J C, Lewis P , et al. Multi-temporal MODIS-Landsat data fusion for relative radiometric normalization,gap filling, and prediction of Landsat data[J]. Remote Sensing of Environment, 2008,112(6):3112-3130.
doi: 10.1016/j.rse.2008.03.009
[41]
Wang P J, Gao F, Masek J G . Operational data fusion framework for building frequent landsat-like imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014,52(11):7353-7365.
doi: 10.1109/TGRS.2014.2311445
Li D C, Tang P, Hu C M , et al. Spatial-temporal fusion algorithm based on an extended semi-physical model and its preliminary application[J]. Journal of Remote Sensing, 2014,18(2):307-319.
[43]
Walker J J, De Beurs K M,Wynne R H,et al.Evaluation of Landsat and MODIS data fusion products for analysis of dryland forest phenology[J]. Remote Sensing of Environment, 2012,117:381-393.
doi: 10.1016/j.rse.2011.10.014
[44]
Liu H, Weng Q H.Enhancing temporal resolution of satellite imagery for public health studies:A case study of West Nile Virus outbreak in Los Angeles in 2007[J]. Remote Sensing of Environment, 2012,117:57-71.
doi: 10.1016/j.rse.2011.06.023
[45]
Singh D . Generation and evaluation of gross primary productivity using Landsat data through blending with MODIS data[J]. International Journal of Applied Earth Observation and Geoinformation, 2011,13(1):59-69.
doi: 10.1016/j.jag.2010.06.007
[46]
Singh D . Evaluation of long-term NDVI time series derived from Landsat data through blending with MODIS data[J]. Atmósfera, 2012,25(1):43-63.
Yin X L, Zhang L, Xu J Y , et al. Application of fused data to grassland biomass estimation[J]. Remote Sensing for Land and Resources, 2013,25(4):147-154.doi: 10.6046/gtzyyg.2013.04.24.
[48]
Yang D, Su H B, Yong Y, et al. MODIS-Landsat data fusion for estimating vegetation dynamics:A case study for two ranches in west texas [C]//1st International Electronic Conference on Remote Sensing.Online, 2015: d016.
Kang J, Wang L, Niu Z , et al. A spatial and temporal fusion model using local spatial association analysis method[J]. Remote Sensing Technology and Application, 2015,30(6):1176-1181.
[50]
Shen H F, Wu P H, Liu Y L , et al. A spatial and temporal reflectance fusion model considering sensor observation differences[J]. International Journal of Remote Sensing, 2013,34(12):4367-4383.
doi: 10.1080/01431161.2013.777488
[51]
Weng Q H, Fu P, Gao F . Generating daily land surface temperature at Landsat resolution by fusing Landsat and MODIS data[J]. Remote Sensing of Environment, 2014,145:55-67.
doi: 10.1016/j.rse.2014.02.003
[52]
Wu M Q, Zhang X Y, Huang W J , et al. Reconstruction of daily 30 m data from HJ CCD,GF-1 WFV,landsat,and MODIS data for crop monitoring[J]. Remote Sensing, 2015,7(12):16293-16314.
doi: 10.3390/rs71215826
[53]
Michishita R, Chen L F, Chen J , et al. Spatiotemporal reflectance blending in a wetland environment[J]. International Journal of Digital Earth, 2015,8(5):364-382.
doi: 10.1080/17538947.2014.894146
[54]
Zhang F, Zhu X L, Liu D S . Blending MODIS and Landsat images for urban flood mapping[J]. International Journal of Remote Sensing, 2014,35(9):3237-3253.
doi: 10.1080/01431161.2014.903351
[55]
Walker J J,de Beurs K M,Wynne R H.Dryland vegetation phenology across an elevation gradient in Arizona,USA,investigated with fused MODIS and Landsat data[J]. Remote Sensing of Environment, 2014,144:85-97.
doi: 10.1016/j.rse.2014.01.007
[56]
Wu P H, Shen H F, Zhang L P , et al. Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature[J]. Remote Sensing of Environment, 2015,156:169-181.
doi: 10.1016/j.rse.2014.09.013
[57]
Zhang B H, Zhang L, Xie D , et al. Application of synthetic NDVI time series blended from Landsat and MODIS data for grassland biomass estimation[J]. Remote Sensing, 2015,8(1):10.
doi: 10.3390/rs8010010
[58]
Dong T F, Liu J G, Qian B D , et al. Estimating winter wheat biomass by assimilating leaf area index derived from fusion of Landsat-8 and MODIS data[J]. International Journal of Applied Earth Observation and Geoinformation, 2016,49:63-74.
doi: 10.1016/j.jag.2016.02.001
[59]
Wu B, Huang B, Cao K , et al. Improving spatiotemporal reflectance fusion using image inpainting and steering kernel regression techniques[J]. International Journal of Remote Sensing, 2017,38(3):706-727.
doi: 10.1080/01431161.2016.1271471
[60]
Zhu X L, Helmer E H, Gao F , et al. A flexible spatiotemporal method for fusing satellite images with different resolutions[J]. Remote Sensing of Environment, 2016,172:165-177.
doi: 10.1016/j.rse.2015.11.016
[61]
Yang J C, Wright J, Huang T S , et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010,19(11):2861-2873.
doi: 10.1109/TIP.2010.2050625
pmid: 20483687
[62]
Huang B, Song H H . Spatiotemporal reflectance fusion via sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012,50(10):3707-3716.
doi: 10.1109/TGRS.2012.2186638
[63]
Song H H, Huang B . Spatiotemporal satellite image fusion through one-pair image learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013,51(4):1883-1896.
doi: 10.1109/TGRS.2012.2213095
[64]
Teillet P M, Fedosejevs G, Thome K J , et al. Impacts of spectral band difference effects on radiometric cross-calibration between satellite sensors in the solar-reflective spectral domain[J]. Remote Sensing of Environment, 2007,110(3):393-409.
doi: 10.1016/j.rse.2007.03.003
Zhong B, Liu Q H, Shan X J , et al. Normalization Processing Technology of Multi-Source Optical Remote Sensing Data[M]. Beijing: Science Press, 2015.
[66]
Chen X X, Vierling L, Deering D . A simple and effective radiometric correction method to improve landscape change detection across sensors and across time[J]. Remote Sensing of Environment, 2005,98(1):63-79.
doi: 10.1016/j.rse.2005.05.021
Yu X M, Zou Q . Methods of radiometric normalization for multi-temporal remote sensing images:A review[J]. Geomatics and Spatial Information Technology, 2012,35(6):8-12.
[68]
Hong G, Zhang Y . A comparative study on radiometric normalization using high resolution satellite images[J]. International Journal of Remote Sensing, 2008,29(2):425-438.
doi: 10.1080/01431160601086019
Geng L Y, Ma M G . Advance in method comparison of reconstructing remote sensing time series data sets[J]. Remote Sensing Technology and Application, 2014,29(2):362-368.
[70]
Gallo K, Ji L, Reed B , et al. Multi-platform comparisons of MODIS and AVHRR normalized difference vegetation index data[J]. Remote Sensing of Environment, 2005,99(3):221-231.
doi: 10.1016/j.rse.2005.08.014
[71]
Leeuwen W J D V,Orr B J, Marsh S E,et al.Multi-sensor NDVI data continuity:Uncertainties and implications for vegetation monitoring applications[J]. Remote Sensing of Environment, 2006,100(1):67-81.
doi: 10.1016/j.rse.2005.10.002
[72]
Xiao X M, Boles S, Frolking S , et al. Mapping paddy rice agriculture in South and Southeast Asia using multi-temporal MODIS images[J]. Remote Sensing of Environment, 2006,100(1):95-113.
doi: 10.1016/j.rse.2005.10.004
[73]
Brown J C, Kastens J H, Coutinho A C , et al. Classifying multiyear agricultural land use data from Mato Grosso using time-series MODIS vegetation index data[J]. Remote Sensing of Environment, 2013,130:39-50.
doi: 10.1016/j.rse.2012.11.009
[74]
Moran M S, Clarke T R, Inoue Y , et al. Estimating crop water deficit using the relation between surface-air temperature and spectral vegetation index[J]. Remote Sensing of Environment, 1994,49(3):246-263.
doi: 10.1016/0034-4257(94)90020-5
Zhao C J . Advances of research and application in remote sensing for agriculture[J]. Transactions of the Chinese Society for Agricultural Machinery, 2014,45(12):277-293.
[76]
Haboudane D, Miller J R, Tremblay N , et al. Integrated narrow-band vegetation indices for prediction of crop chlorophyll content for application to precision agriculture[J]. Remote Sensing of Environment, 2002,81(2-3):416-426.
doi: 10.1016/S0034-4257(02)00018-4
[77]
Liu L, Wang J, Bao Y , et al. Predicting winter wheat condition,grain yield and protein content using multi-temporal EnviSat-ASAR and Landsat TM satellite images[J]. International Journal of Remote Sensing, 2006,27(4):737-753.
doi: 10.1080/01431160500296867
[78]
Zhong L H, Gong P, Biging G S . Efficient corn and soybean mapping with temporal extendability:A multi-year experiment using Landsat imagery[J]. Remote Sensing of Environment, 2014,140:1-13.
doi: 10.1016/j.rse.2013.08.023