Title:	Advancing the Low-resource Regimes with Generative AI
DNr:	Berzelius-2025-114
Project Type:	LiU Berzelius
Principal Investigator:	Xu Guo <xugu@kth.se>
Affiliation:	Kungliga Tekniska högskolan
Duration:	2025-04-29 – 2025-11-01
Classification:	10208
Homepage:	https://guoxuxu.github.io/
Keywords:

Abstract

The recent success of Artificial Intelligence (AI) comes with a variety of Large Pretrained Models (LPMs), which excelled state-of-the-art technologies on a range of benchmarks such as GLUE and superGLUE. However, it requires sufficient data and computing budgets to harness the power of LPMs. In fact, many low-resource tasks, especially in healthcare and finance, have not enjoyed the benefits of AI as much as high-resource tasks, because collecting data is quite expensive and even impractical in those scenarios. Traditional research for low-resource tasks is dedicated to developing machine learning-based methods, e.g., transfer learning and parameter-efficient tuning, leaving the fundamental shortage of training data unresolved. This proposal aims to transform the low-resource regimes by eliminating the demand for labeled data via LPMs. Inspired by recent studies that Pretrained Language Models (PLMs) can generate meaningful data when guided by prompts, we propose to distill training data from PLMs to enrich low-resource tasks. The goal is to allow downstream tasks, regardless of resource constraints, to equally benefit from AI, and thereby accelerating Artificial General Intelligence (AGI). We will provide a complete set of methodologies for PLM-based data generation and usage to improve low-resource learning performance. TL;DR - My WASP project aims to develop new data-/compute- efficient algorithms to translate the generative power of Large Language Models to advance downstream real-world problems.