BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Diagnosing what is in a Language Model: On the Pitfalls of Probes 
 and Prompts
DTSTART:20221111T150000
DTEND:20221111T170000
DTSTAMP:20261005T022001Z
UID:cbbdc062f5f071355c59a997ac817d40fef1a1fbf286b076fb24228c
CATEGORIES:Conferences - Seminars
DESCRIPTION:Deniz Bayazit\nEDIC candidacy exam\nExam president: Prof. Boi 
 Faltings\nThesis advisor: Prof. Antoine Bosselut\nCo-examiner: Prof. Marti
 n Jaggi\n\nAbstract\nProbing and prompting results show that pre-trained l
 anguage models can encode linguistic and factual properties. These approac
 hes make assumptions\, both at the probing and prompting level\, that fund
 amentally affect the conclusions on the abilities of language models.\nIn 
 this proposal\, to verify this hypothesis\, we first investigate how probe
 s can memorize linguistic tasks through supervision. Then\, we examine how
  different prompting schemes may overfit the prediction distribution to a 
 standard dataset's golden answer distribution. Finally\, we review a propo
 sitional logic augmentation to language model prompting that can infer rob
 ust and consistent answers.\n\nFollowing these works on mitigating the pit
 falls of probing and prompting\, we propose developing behavioral diagnost
 ic tools that can more robustly provide insights into the encoded properti
 es of language models.\n\nBackground papers\nDesigning and Interpreting Pr
 obes with Control Tasks\nhttps://aclanthology.org/D19-1275/ \nEMNLP 2019\
 n\nKnowledgeable or Educated Guess? Revisiting Language Models as Knowledg
 e Bases\nhttps://aclanthology.org/2021.acl-long.146/ \nACL 2021\n\nMaieut
 ic Prompting: Logically Consistent Reasoning with Recursive Explanations\n
 https://arxiv.org/abs/2205.11822 \nEMNLP 2022\n\n 
LOCATION:BC 233 https://plan.epfl.ch/?room==BC%20233
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR