DL4NLP: Deep Learning for Natural Language Processing
Abstract
The field of natural language processing (NLP) has enjoyed significant breakthroughs in the last few years with the development of deep neural language models such as BERT and GPT-3. When trained on large amounts of text data, these models are able to learn distributed representations of language meaning, which have been successfully used in a wide range of language understanding tasks.
The NLP Group is currently working on three different research tasks related to deep neural language models: 1. interpreting pre-trained representations; 2. grounding these representations in non-linguistic modalities such as images and video; 3. connecting pre-trained representations to structured representations such as knowledge graphs. All of these tasks require large amounts of labelled training data and significant GPU-resources. Until now we have been distributing our resource needs over several clusters both within and outside of the SNIC infrastructure. With the present project, we want to start consolidating our compute resources to a single cluster. We start with the default resource allocation but expect to up-scale our requirements after the first project phase, as we are expanding our group and move projects from other clusters to BerzeLiUs.
