Title:	Motion inbetweening with diffusion models
DNr:	Berzelius-2024-344
Project Type:	LiU Berzelius
Principal Investigator:	Rajmund Nagy <rajmundn@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2024-09-10 – 2025-04-01
Classification:	10201
Keywords:

Abstract

Animation sequences in video games and movies are often created in a hierarchical manner, starting with keyframes, which are in a sense the most important moments of the sequence, and then filling in the gaps between them. This process, known as (motion) inbetweening, is an especially tedious part of character animation, with great potential for automation. Score-based diffusion models have proven to be powerful generative models not only for images and audio, but also for 3D motion capture data used in character animation. Several strong models have been developed for text- or audio-driven motion generation purposes over recent years, but incorporating sparse keyframe-conditioning into these models to tackle inbetweening tasks remains an actively studied problem. This project aims to address key limitations of state-of-the-art motion inbetweening models – e.g., slow generation process and inability to handle long gaps or dynamic movements – by incorporating stronger generative models (e.g., flow matching), heterogeneous datasets, and novel frame masking procedures during training.