Title:	Graph-based, spatial, temporal, and generative machine learning
DNr:	Berzelius-2024-368
Project Type:	LiU Berzelius
Principal Investigator:	Fredrik Lindsten <fredrik.lindsten@liu.se>
Affiliation:	Linköpings universitet
Duration:	2024-10-01 – 2025-04-01
Classification:	10201
Keywords:

Abstract

Development and Evaluation of Neural Limited Area Weather Forecasting Models: Weather forecasting is crucial for society, and many institutes and companies invest large efforts into producing accurate and timely forecasts. It is often of interest to produce forecasts for a specific region, and limited area models are one way to achieve this. As new machine learning techniques are being applied to weather forecasting, there is substantial interest to use these also for regional forecasting. In this project we are building on our previous pioneering work within the area [1] to develop such neural limited area models applicable to more realistic settings. We will investigate the different design choices in these models and use these insights to build accurate models for multiple different regions. The weather models will operate on fine spatial resolutions, requiring large computational resources. In this project we have multiple collaborators from SMHI, Danish Meteorological Institute, ETH Zurich and GeoSphere Austria. To make the outcome of the research useful for practitioners we aim to publish this work in the earth system modeling literature and make the tools developed throughout the project openly available. By: Joel Oskarsson [1] Oskarsson, J., Landelius, T. and Lindsten, F., 2023. Graph-based Neural Weather Prediction for Limited Area Modeling. NeurIPS 2023 Workshop on Tackling Climate Change with Machine Learning. ------ Probabilistic methods in machine learning weather prediction: Weather is inherently stochastic and a full treatment of the weather forecasting problem requires a probabilistic approach. Machine learning and in particular generative modeling have shown promising results in this regard. In this project we aim to build on these ideas to develop new ways of generating probabilistic forecasts. In particular, we will consider the general problem of learning to propagate probability distributions through time by learning the relevant dynamics. This also includes generalizing the denoising diffusion objective to better suit the temporal nature of weather. By: Martin Andrae ------ Improving roll out stability of machine learning weather prediction (MLWP): As most MLWP models are trained on making predictions on a fixed time scale and then rolled out autoregressively to make longer forecasts errors accumulate, making them unreliable on longer forecast horizons. In this project we aim to improve the stability of the rollouts for MLWP models by investigating the model design as well as the training and inference paradigm. Conditioned limited area weather forecasting: When using a model for making regional limited area forecasts can be conditioned on information given from a global weather prediction model to generate high resolution forecasts on a regional scale which might not be possible on a global scale. This modular approach allows for more flexibility in the model design for each region while incorporating information from the global forecast. By: Erik Larsson ------ Denoising Diffusion-based Sequential Monte Carlo Sampler: Denoising diffusion models are a class of generative models known for their state-of-the-art performance in tasks such as image synthesis, video generation, and related applications. Their core principle involves employing a noise diffusion process to gradually transform a complex data distribution into a Gaussian distribution. Samples from diffusion models are generated by approximating the time-reversal of this diffusion process, typically initialized with Gaussian noise. We aim to apply this concept to sample approximately from a given target distribution. Specifically, we consider a noise diffusion process where the target distribution gradually diffuses towards a Gaussian distribution. However, the widely used score matching technique is not applicable in this context, as sampling directly from the target distribution is impossible by assumption. Therefore, we aim to integrate the denoising diffusion approach with the sequential Monte Carlo (SMC) algorithm, drawing on ideas from [1, 2]. [1] Francisco Vargas, et al. Denoising Diffusion Samplers. ICLR, 2023. [2] Angus Phillips, et al. Particle Denoising Diffusion Sampler. ICML, 2024. By: Dong Qian ----- Generative modeling for discrete diffusion models: Generative modeling has seen a tremendous interest lately, with so called diffusion models showing impressive generative performance and capabilities. A lot of focus has been put on continuous data as images, but lately also discrete models have been developed. As discrete data has significantly different properties compared to continuous data, designing diffusion generative models for discrete data requires special care, and designated methods for this type of data. This project aims at developing methods for diffusion models for discrete data, with an emphasis on so called conditional generation, where gradient-based methods from the continuous case are not readily available. This will partly be part of the WASP-WISE project "Generative AI models for property to structure materials prediction", funded by the Knut and Alice Wallenberg Foundation. By: Filip Ekström Kelvinius ----- Coupling diffusion models with Bayesian inference for improved generation, inference, and likelihood computation: Conceptually, training a generative model is very similar to a conventional statistical learning problem, where a parametric model is fitted to observed training data. Once the model parameters have been learned, samples can be generated by simulating from the parametric model. This naturally leads to the questions – Can we leverage recent advances in generative AI for solving conventional statistical problems? Can we leverage state-of-the-art statistical inference methods for improving generative modelling? This project sets out to answer these questions, in a constructive manner by deriving novel machine learning methods for tackling the methodological problems that underlie the two questions. By: Adhithyan Kalaivanan