Title:	Deep Learning for 3D Computer Vision using Geometric Information and Generative Models
DNr:	Berzelius-2024-338
Project Type:	LiU Berzelius
Principal Investigator:	Fredrik Kahl <fredrik.kahl@chalmers.se>
Affiliation:	Chalmers tekniska högskola
Duration:	2024-09-28 – 2025-04-01
Classification:	10207
Homepage:	https://neural3d.github.io/
Keywords:

Abstract

This is a continuation project with the same goal as the previous one. We are still working on the same research directions. For detailed updates, see description of "Resource Usage". The overall goal is to perform research on applying Deep Learning to problems within 3D Computer Vision, finding ways to use neural networks to from images and videos learn how to reason about the world in 3D. This enables a wide array of applications including autonomous navigation, AR and biomedical imaging. We will look at the fundamental task of estimating relative motion between two images using deep learning. This is a prerequisite for downstream applications such as novel view synthesis. Furthermore, recently generative methods such as Diffusion Models have enabled using powerful networks pre-trained on extremely large amounts of data. We will in this project look at how these Diffusion Models can be used to improve performance in 3D tasks such as novel view synthesis, focusing on dynamic scenes and improving geometric consistency. A goal is to develop methods that utilize geometric information to improve performance in an efficient way, without requiring extensive training procedures. The project consists of two PhD students and one postdoc.