Analysis of Transformer Language Models at Finer Granularities
Event details
| Date | 25.08.2022 |
| Hour | 14:00 › 16:00 |
| Speaker | Deniz Bayazit |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Boi Faltings
Thesis advisor: Prof. Antoine Bosselut
Co-examiner: Prof. Martin Jaggi
Abstract
Pre-trained language models (LMs), particularly Transformer-based ones, achieve good performance on knowledge-related tasks, hinting that such models are able to encode and manipulate structured information depicted by natural language.
However, we do not have a complete grasp of the internal processes of LMs and what part of the network is responsible for this ability. Recent work decoupling Transformer processes often focus on a surface-level granularity of the model, making it harder to structurally probe and edit LMs with confidence.
In this proposal, we will show 3 different granularities of analysis done in the past: (1) the layer level, (2) the neuron level, and (3) the weight level. At each level, we consider how the methods and findings push our understanding of how structured information in natural language is processed by Transformer-based LMs.
We argue that finding structured information encoding subnetworks in LMs can allow us to manipulate them. Consequently, we propose a study on locating subnetworks of parameters responsible with encoding conceptual knowledge within LMs.
Background papers
Exam president: Prof. Boi Faltings
Thesis advisor: Prof. Antoine Bosselut
Co-examiner: Prof. Martin Jaggi
Abstract
Pre-trained language models (LMs), particularly Transformer-based ones, achieve good performance on knowledge-related tasks, hinting that such models are able to encode and manipulate structured information depicted by natural language.
However, we do not have a complete grasp of the internal processes of LMs and what part of the network is responsible for this ability. Recent work decoupling Transformer processes often focus on a surface-level granularity of the model, making it harder to structurally probe and edit LMs with confidence.
In this proposal, we will show 3 different granularities of analysis done in the past: (1) the layer level, (2) the neuron level, and (3) the weight level. At each level, we consider how the methods and findings push our understanding of how structured information in natural language is processed by Transformer-based LMs.
We argue that finding structured information encoding subnetworks in LMs can allow us to manipulate them. Consequently, we propose a study on locating subnetworks of parameters responsible with encoding conceptual knowledge within LMs.
Background papers
- Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space https://arxiv.org/abs/2203.14680, Preprint on arxiv.
- Analyzing Individual Neurons in Pre-trained Language Models https://aclanthology.org/2020.emnlp-main.395/ , EMNLP 2020
- Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks https://openreview.net/forum?id=7uVcpu-gMD, ICLR 2021
Practical information
- General public
- Free