Title:	Generative AI for Autonomous Driving Scene Generation and Reconstruction
DNr:	Berzelius-2025-119
Project Type:	LiU Berzelius
Principal Investigator:	Fredrik Kahl <fredrik.kahl@chalmers.se>
Affiliation:	Chalmers tekniska högskola
Duration:	2025-04-06 – 2025-11-01
Classification:	10207
Homepage:	https://neural3d.github.io/
Keywords:

Abstract

This project provides GPU access to many ongoing WASP funded research projects within generative methods applied to autonomous driving challenges. A few examples of projects actively being worked on as well as planned for the near future are: 1) Generative image and video diffusion models for fixing artifacts from 3D gaussian splatting (3DGS) and Neural renderings (NeRFs) in autonomous driving scenes, 2) Multimodal Large Models for editing real driving data to create realistic edge case scenarios 3) World Foundation Models to generate novel driving 4D scene from text or road markers (intended for use in closed loop simulation and testing) 4) Creation of 4D driving scene dataset for closed loop simulation and testing of end-to-end driving systems 5) Use of regularization methods to improved 3D gaussian splatting of dynamic driving scenes These applications motivate our need for GPUs. All these projects together address the inevitable problem of lack of diverse autonomous driving data and limited ability to simulate every edge case and dangerous scenario. On average, a person (in the US) experiences 3 to 4 car accidents in a lifetime and travels 24 million kilometers, which shows how infeasible it is to rely on test cars acquiring limited amount of data expecting to have a good representation of all driving scenarios, particularly edge cases and dangerous situations. The usual methods of testing these with simulation or controlled scenarios with dummy actors are insufficient and limited by not only resources and time but our own imagination in designing them. By instead using generative methods, we can create rare scenarios, craft never recorded data and modify and expand existing data to train and validate autonomous driving systems. The project will involve the PI and two new WASP PhD students that started in 2025.