Climate Prediction in Reduced Dimensions: A Comparative Analysis of Reservoir Computing and Attention-based Transformers

This study focuses on the application of machine learning techniques to better characterize predictability of the spatiotemporal variability of sea surface temperature (SST) on the basin scale. Both, sub-seasonal variability including extreme events (cf. marine heatwaves) and interannual variability are considered. 

We rely on dimensionality reduction techniques---linear principal component analysis (PCA)  and nonlinear Variational Autoencoders (VAE)---to then perform the actual prediction tasks in the corresponding latent space using two disparate methodologies: reservoir computing (RC), and attention-based transformers. 

After comparing performance, we examine various issues including the role of generalized synchronization in RC and implicit memory of RC vs. explicit long-term memory of transformers with the broad aim of shedding light on the effectiveness of these techniques in the context of data-driven climate prediction.