Title:	Deep Learning for the Physical World
DNr:	Berzelius-2025-339
Project Type:	LiU Berzelius
Principal Investigator:	Mårten Björkman <celle@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2025-12-29 – 2026-07-01
Classification:	10210
Keywords:

Abstract

This proposal involves two separate research tracks that both explore deep learning for representation of complex real-world image data. In the first track, we study the geometric structure that underpins neural network representations and design algorithms to explicitly control it to enhance the models' generalization and robustness. The second track applies geometry-aware representation learning to the problem of deformable object reconstruction. By leveraging physics-based simulation methods, the project aims to enhance current reconstruction approaches, by incorporating prior knowledge of the dynamics and deformability of the objects. Track 1: Understanding Learning and Memorization in Deep Networks on Natural Image Data Deep networks achieve state-of-the-art performance on several tasks within the natural world, showing a remarkable generalization ability, despite training sets that appear too small in relation to the capacity of networks. This track investigates mechanisms underpinning this ability and the role played by the network architecture and the optimization procedure for training the network. Of particular interest is the study of robustness emerging from large-scale training, vis-à-vis the geometry underpinning neural representations. Extending prior work from the research group, the project proposes interpretable methods for provably modulating the geometry of a model's decision boundaries, thereby provably controlling generalization and robustness. Track 2: Neural representation and rendering Existing works on deformable object reconstruction usually incorporate information exclusively from the currently observed scene, limiting them in regard to processing times, sparse observations and complicated deformations. This project intends to overcome these limitations by exploring physics-based simulation methods, which, instead of being optimized on a single scene, make use of prior knowledge about object dynamics and deformations.The special focus of this project is on highly deformable objects such as rope-like or cloth-like objects for which we want to explore the usage of both explicit physics simulation (MPM, PDB), and learned approaches (GNN) to model their dynamics.