Diagnosing what is in a Language Model: On the Pitfalls of Probes and Prompts
Event details
| Date | 11.11.2022 |
| Hour | 15:00 › 17:00 |
| Speaker | Deniz Bayazit |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Boi Faltings
Thesis advisor: Prof. Antoine Bosselut
Co-examiner: Prof. Martin Jaggi
Abstract
Probing and prompting results show that pre-trained language models can encode linguistic and factual properties. These approaches make assumptions, both at the probing and prompting level, that fundamentally affect the conclusions on the abilities of language models.
In this proposal, to verify this hypothesis, we first investigate how probes can memorize linguistic tasks through supervision. Then, we examine how different prompting schemes may overfit the prediction distribution to a standard dataset's golden answer distribution. Finally, we review a propositional logic augmentation to language model prompting that can infer robust and consistent answers.
Following these works on mitigating the pitfalls of probing and prompting, we propose developing behavioral diagnostic tools that can more robustly provide insights into the encoded properties of language models.
Background papers
Designing and Interpreting Probes with Control Tasks
https://aclanthology.org/D19-1275/
EMNLP 2019
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
https://aclanthology.org/2021.acl-long.146/
ACL 2021
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
https://arxiv.org/abs/2205.11822
EMNLP 2022
Exam president: Prof. Boi Faltings
Thesis advisor: Prof. Antoine Bosselut
Co-examiner: Prof. Martin Jaggi
Abstract
Probing and prompting results show that pre-trained language models can encode linguistic and factual properties. These approaches make assumptions, both at the probing and prompting level, that fundamentally affect the conclusions on the abilities of language models.
In this proposal, to verify this hypothesis, we first investigate how probes can memorize linguistic tasks through supervision. Then, we examine how different prompting schemes may overfit the prediction distribution to a standard dataset's golden answer distribution. Finally, we review a propositional logic augmentation to language model prompting that can infer robust and consistent answers.
Following these works on mitigating the pitfalls of probing and prompting, we propose developing behavioral diagnostic tools that can more robustly provide insights into the encoded properties of language models.
Background papers
Designing and Interpreting Probes with Control Tasks
https://aclanthology.org/D19-1275/
EMNLP 2019
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
https://aclanthology.org/2021.acl-long.146/
ACL 2021
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
https://arxiv.org/abs/2205.11822
EMNLP 2022
Practical information
- General public
- Free