Title:	Robot Navigation using Foundation Models
DNr:	Berzelius-2025-135
Project Type:	LiU Berzelius
Principal Investigator:	Finn Lukas Busch <flbusch@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2025-04-04 – 2025-11-01
Classification:	20208
Homepage:	https://www.finnbusch.com/OneMap/
Keywords:

Abstract

Autonomous robots operating in unknown environments face significant challenges in navigation and decision-making. Traditional approaches rely heavily on dense geometric mapping and localization, which can be computationally expensive and brittle in novel settings, and often are not alined with higher-level language described goals. In contrast to that, humans can build a "lightweight" map of an environment easily, and remember important aspects such that they can navigate the environment reliably. Moreover, they can remember the structure of an environment, and how different areas are connected without the need for a globally consistent, highly accurate dense map. This project aims to enhance robot navigation capabilities by integrating semantic understanding through foundation models, enabling robots to reason about their environment at a conceptual level beyond pure geometry, without the need for dense maps. This allows constructing more abstract, high-level maps of the environment. By recognizing important objects, or landmarks seen during previous exploration the robot can then reliably navigate new environments. To equip robots with these capabilities, we plan to both use off-the-shelf foundation models and train specific solutions in simulation that give robots a general understanding of the world, helping them build sparse maps of the environment in one shot and allowing them to localize, and navigate reliably without the need for dense maps or expensive sensors.