Causal Inference for Creativity in NLP
Event details
| Date | 28.10.2022 |
| Hour | 16:30 › 18:30 |
| Speaker | Molly Petersen |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Robert West
Thesis advisor: Prof. Antoine Bosselut
Thesis co-advisor: Prof. Lonneke van der Plas
Co-examiner: Prof. Maria Brbic
Abstract
Creativity is important in accomplishing a wide variety of tasks, not excluding scientific discovery and problem solving. In science, creativity still must fit into the constraints with how humans have decided to go about scientific reasoning, specifically the concepts that fit under the scientific method. In this candidacy, we cover three papers that cover the topics of causal inference, a commonly employed paradigm in a variety of scientific disciplines, as well as creativity . The first paper we cover presents a paradigm for creating counterfactual text representation, to allow for the identification of the effects of a particular concept exposure of interest on a downstream classification task . Next, we will cover a paper where the authors finetune a language model to generate counterfactual text, which aside from assisting causal inference in NLP, is arguably itself a creative task . Lastly, we will explore a paper that presents different analyses for evaluating novelty in generated text .
Background papers
CausaLM: Causal Model Explanation Through Counterfactual Language Models
https://arxiv.org/pdf/2005.13407.pdf
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
https://arxiv.org/pdf/2111.09509.pdf
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
https://arxiv.org/abs/2101.00288
Exam president: Prof. Robert West
Thesis advisor: Prof. Antoine Bosselut
Thesis co-advisor: Prof. Lonneke van der Plas
Co-examiner: Prof. Maria Brbic
Abstract
Creativity is important in accomplishing a wide variety of tasks, not excluding scientific discovery and problem solving. In science, creativity still must fit into the constraints with how humans have decided to go about scientific reasoning, specifically the concepts that fit under the scientific method. In this candidacy, we cover three papers that cover the topics of causal inference, a commonly employed paradigm in a variety of scientific disciplines, as well as creativity . The first paper we cover presents a paradigm for creating counterfactual text representation, to allow for the identification of the effects of a particular concept exposure of interest on a downstream classification task . Next, we will cover a paper where the authors finetune a language model to generate counterfactual text, which aside from assisting causal inference in NLP, is arguably itself a creative task . Lastly, we will explore a paper that presents different analyses for evaluating novelty in generated text .
Background papers
CausaLM: Causal Model Explanation Through Counterfactual Language Models
https://arxiv.org/pdf/2005.13407.pdf
How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
https://arxiv.org/pdf/2111.09509.pdf
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
https://arxiv.org/abs/2101.00288
Practical information
- General public
- Free
Contact
- edic@epfl.ch