BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:IC Colloquium: Towards understanding the learning dynamics of neur
 al networks
DTSTART:20231116T161500
DTEND:20231116T173000
DTSTAMP:20260408T121841Z
UID:8f03bb454d2ba38414946e84fd2c9e3909a10f4a2f03571daefdb04f
CATEGORIES:Conferences - Seminars
DESCRIPTION:By: Razvan Pascanu - Google DeepMind\nVideo of his talk\n\nAbs
 tract\nIn this topic I will discuss a topic that I've been interested in s
 ince my PhD\, the learning dynamics of neural networks. Specifically I wil
 l try to provide one intuition of why learning tends to be computationally
  and data inefficient for deep learning\, using this as an entry point to 
 introduce the topic of continual learning. The main argument is that inter
 ference between gradients coming from different modes of the data can lead
  to them being learned sequentially even though data needs to be sampled i
 n an IID fashion from the entire distribution. I will summarize how contin
 ual learning might relate to this issue and some of the main themes within
  this growing subfield. One problem  that I would spend a bit more time o
 n is that of plasticity loss which has been a focus in continual reinforce
 ment learning in the last couple of years. If time allows\, I would touch 
 on the topic of generalization\, the main goal of learning\, and in partic
 ular of compositional generalization as well as the concept of alignment\,
  explored more thoroughly within the space of algorithmic reasoning.  \
 n\nBio\nRazvan Pascanu has been a research scientist at Google DeepMind si
 nce 2014. Before this\, he did his PhD in Montréal with prof. Yoshua Beng
 io\, working on understanding deep networks\, recurrent models and optimi
 zation. Since he joined Google DeepMind he has also had significant contri
 butions in deep reinforcement learning\, continual learning\, meta-learnin
 g\, graph neural networks as well as continuing his research agenda of und
 erstanding deep learning\, recurrent models and optimization. Please see h
 is scholar page for specific contributions. He is also actively promotin
 g AI research and education as a main organizer of Conference on Life-long
  Learning Agents (CoLLAs) lifelong-ml.cc \, Eastern European Machine Lear
 ning Summer School (EEML) www.eeml.eu and www.workshops.eeml.eu as wel
 l as different workshops at NeurIPS\, ICML and ICLR.\n\nMore information
LOCATION:BC 420 https://plan.epfl.ch/?room==BC%20420 https://epfl.zoom.us/
 j/63876969961
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR