Title:	Large-Scale Spatio-Temporal Reasoning and Learning
DNr:	Berzelius-2025-74
Project Type:	LiU Berzelius
Principal Investigator:	Fredrik Heintz <fredrik.heintz@liu.se>
Affiliation:	Linköpings universitet
Duration:	2025-03-01 – 2025-09-01
Classification:	10210
Homepage:	https://www.ida.liu.se/~frehe08/
Keywords:

Abstract

The goal of our research is to develop novel reasoning and learning methods for large-scale spatio-temporal applications. This includes for example large-language models, time-series learning (transformers, diffusion models and GANs) and multi-agent reinforcement learning. The expected scientific impact is publications in top-level conferences and the expected soecity impact is more effective decision-making methods for autonomous systems such as unmanned aircraft, more effective transporation solutions and methods for privacy-preserving synthetic data generation. In addition to the current project, we will also investigate foundation world-models which will require a sizable amount of space for data storage. Specifically, foundation world-models which have been shown to be able to simulate interactive 3D environments by learning how different actions change the world. The learned simulated environments can be used to rapidly prototype environments for video games and architecture, as well as a way to safely develop and evaluate autonomous systems. This is achieved by analyzing videos of agents interacting with an environment, optionally annotated with the actions taken. In this project, we aim to investigate and improve the consistency and object permanence of these models. Current state of the art models maintain consistency over a relatively short time horizon, causing the models to eventually lose track of their position and their environments, ultimately limiting their usefulness. Instead of simply expanding the time horizon of these models, we would like to develop a long-term memory architecture for the model to save information across longer time horizons. The memory can be continuously updated, to keep track of changes to the environment and any new information that has been generated, as well as read whenever the model is asked to reproduce locations it has previously generated data for.