Title:	Sim to Real: Automatic manufacturing quality inspection based on computer vision
DNr:	Berzelius-2024-258
Project Type:	LiU Berzelius
Principal Investigator:	Atsuto Maki <atsuto@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2024-08-01 – 2025-02-01
Classification:	10207
Keywords:

Abstract

A challenge to applying deep learning-based computer vision technologies for manufacturing quality inspection lies in the cost, time, and manual effect of collecting a large amount of annotated training data. This project aims to solve this challenge by training a model with only synthetics data generated from manufacturing CAD models. The method should be able to ignore the gap between the simulation and real data. Meaning the model should be able to give satisfying results on real data while only being trained on synthetic data. In the previous Berzelius project, we developed a synthetic data generation pipeline and a continuous training pipeline. The goal of continuous training is to iteratively generate synthetic data, train models, and use the insights gained to enhance subsequent data generation cycles for optimal performance. In this project, we aim to further refine the continuous training pipeline with five research ideas: 1) extending our work to encompass additional manufacturing assembly use cases; 2) improving the continuous training pipeline; 3) exploring the incorporation of language-vision models to enhance sim-to-real object detection; 4) incorporating depth information to enhance sim-to-real object detection; 5) investigating video applications using self-supervised learning techniques. This project is related to Scania CV AB and KTH, and partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Marianne and Marcus Wallenberg Foundation. In addition, Jacob from Uppsala University will work with Scania on this project in the next six months.