README.md

Title: Enhanced wavelet based spatiotemporal fusion networks using cross-paired remote sensing images [paper]

X. Zhang, S. Li, Z. Tan, et al, "Enhanced wavelet based spatiotemporal fusion networks using cross-paired remote sensing images," ISPRS Journal of Photogrammetry and Remote Sensing, Volume. 211, pp. 281-297, May, 2024.

Abstract: Spatiotemporal fusion can provide remote sensing images with both high temporal and high spatial resolution for earth observation applications. Most of the state-of-the-art models require three or even five images as input, which may lead to difficulties in practical applications due to bad weather or data missing. In this paper, the enhanced cross-paired wavelet based spatiotemporal fusion networks (ECPW-STFN) are proposed to improve the accuracy and quality of the fusions with fewer remote sensing images as inputs. Wavelet transform is introduced into spatiotemporal image fusion to achieve separate training of the high and low frequency components of the image to better extract features of different levels. In addition, a compound loss function containing wavelet loss is proposed to facilitate the preservation of details. Also, an enhancement module with convolutional block attention is put forward to further refine the prediction quality. Experiments were conducted to compare the proposed ECPW-STFN with five state-of-the-art methods on the public CIA and Daxing datasets. The results show ECPW-STFN is better than GAN-STFM which also requires two images, and not inferior to the methods requiring at least three inputs, even exceeds the optimal MLFF-GAN in some cases, proving its great superiority and potential.

Fig. 1. Overall structure of proposed ECPW-STFN model (F and C denote the fine-resolution image and the coarse-resolution image, respectively; the subscripts “ref” and “pred” denote the corresponding image is at reference date and prediction date, respectively).