BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Two Analyses of Modern Deep Learning: Graph Neural Networks and La
 nguage Model Finetuning
DTSTART:20240227T111500
DTEND:20240227T121500
DTSTAMP:20260510T211239Z
UID:cd141b30318ef6b949666de4c6a95a0b6a15bcbfec320ed7f2fd2955
CATEGORIES:Conferences - Seminars
DESCRIPTION:Noam Razin (Tel Aviv University)\nThe resurgence of deep lear
 ning was largely driven by architectures conceived in the 20th century tra
 ined using labeled data. In recent years deep learning has undergone parad
 igm shifts characterized by new architectures and training regimes. Despit
 e the popularity of the new paradigms their theoretical understanding is l
 imited. In this talk I will present two recent works analyzing aspects of 
 modern deep learning. The first work considers the expressive power of gra
 ph neural networks and formally quantifies their ability to model interact
 ions between vertices. As a practical application of the theory I will int
 roduce a simple edge sparsification algorithm that achieves state-of-the-a
 rt results. The second work identifies a fundamental vanishing gradients p
 roblem that occurs when using reinforcement learning to finetune language 
 models. I will demonstrate the detrimental effects of this phenomenon and 
 present possible solutions. Lastly I will conclude with an outlook on impo
 rtant questions raised by the advent of foundation models and possible too
 ls for addressing them.\n\nWorks covered in the talk were in collaboration
  with Nadav Cohen Tom Verbin Hattie Zhou Omid Saremi Vimal Thilak Arwen Br
 adley Preetum Nakkiran Joshua Susskind and Etai Littwin.
LOCATION:GA 3 21 https://plan.epfl.ch/?room==GA%203%2021
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR