On the Creativity of Language Models
Event details
| Date | 16.07.2024 |
| Hour | 11:00 › 13:00 |
| Speaker | Mahammad Ismayilzada |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Pearl Pu
Thesis advisor: Prof. Antoine Bosselut
Thesis co-advisor: Prof Lonneke van der Plas
Co-examiner: Prof. Sabine Süsstrunk
Abstract
Language models, especially, large language models have revolutionized a broad range of fields in Artificial Intelligence from natural language processing to computer vision. Their seemingly remarkable human-like language generation and reasoning abilities have also led people to use them for various creative purposes such as creative writing, poetry, and problem-solving. However, the current language models have also been shown to lack strong generalization and "out-of-the-box" thinking capabilities which are often considered essential prerequisites for true creativity. This discrepancy raises several important questions: Can language models produce truly novel and diverse content and solve problems creatively like humans? In this work, we survey the literature to shed some light on these questions drawing from the research investigating problem-solving, linguistic generalization, and diverse content generation abilities of language models.
Background papers
Exam president: Prof. Pearl Pu
Thesis advisor: Prof. Antoine Bosselut
Thesis co-advisor: Prof Lonneke van der Plas
Co-examiner: Prof. Sabine Süsstrunk
Abstract
Language models, especially, large language models have revolutionized a broad range of fields in Artificial Intelligence from natural language processing to computer vision. Their seemingly remarkable human-like language generation and reasoning abilities have also led people to use them for various creative purposes such as creative writing, poetry, and problem-solving. However, the current language models have also been shown to lack strong generalization and "out-of-the-box" thinking capabilities which are often considered essential prerequisites for true creativity. This discrepancy raises several important questions: Can language models produce truly novel and diverse content and solve problems creatively like humans? In this work, we survey the literature to shed some light on these questions drawing from the research investigating problem-solving, linguistic generalization, and diverse content generation abilities of language models.
Background papers
- MacGyver: Are Large Language Models Creative Problem Solvers? (NAACL 2024), Yufei Tian, Abhilasha Ravichander, Lianhui Qin, Ronan Le Bras, Raja Marjieh, Nanyun Peng, Yejin Choi, Thomas L. Griffiths, Faeze Brahman
- How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN (TACL 2023),
R. Thomas McCoy, Paul Smolensky, Tal Linzen, Jianfeng Gao, Asli Celikyilmaz - Does Writing with Language Models Reduce Content Diversity? (ICLR 2024), Vishakh Padmakumar, He He
Practical information
- General public
- Free