DL4NLP: Deep Learning for Natural Language Processing
Abstract
The field of natural language processing (NLP) has enjoyed significant breakthroughs in the last few years with the development of deep neural language models such as BERT and GPT-3. When trained on large amounts of text data, these models are able to learn distributed representations of language meaning, which have been successfully used in a wide range of language understanding tasks.
The NLP Group is currently working on three different research tasks related to deep neural language models: 1. interpreting pre-trained representations; 2. grounding these representations in non-linguistic modalities such as images and video; 3. connecting pre-trained representations to structured representations such as knowledge graphs.
Each of these tasks requires large amounts of compute, especially GPU resources. Before the start of this project, we were distributing our resource needs over several clusters both within and outside of the SNIC infrastructure. With the present project, we started consolidating resources and transferring them to the Berzelius cluster.
