BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:CIS - Colloquium -  by Prof. Surya Ganguli\, Stanford
DTSTART:20221003T180000
DTEND:20221003T190000
DTSTAMP:20260917T140735Z
UID:641ea608b3b0d60036736cbc83ad1c815721f9e29367c29f85860c84
CATEGORIES:Conferences - Seminars
DESCRIPTION:  Prof Surya Ganguli\nTitle: From statistical physics theory 
 to machine learning algorithms: how to beat neural scaling laws through da
 ta pruning \n\nAbstract:\nStatistical mechanics theory and neural network
  experiments have long enjoyed fruitful interactions spanning the fields o
 f neuroscience and machine learning alike.  These interactions have provi
 ded both conceptual insights into neural network function as well as engin
 eering insights into how to improve network performance. We will review so
 me of our recent work in this area and then focus on one recent story invo
 lving neural scaling laws and how to beat them.  Neural scaling experimen
 ts reveal that the error of many neural networks falls off as a power law 
 with network size\, dataset size or compute.  Such power laws have motiva
 ted significant societal investments in large scale model training and dat
 a collection efforts.  Inspired by statistical mechanics calculations\, w
 e show both in theory and in practice how we can beat neural power law sca
 ling with respect to dataset size\, sometimes achieving much better expone
 ntial scaling instead\, by collecting small carefully curated datasets rat
 her than large random ones.  This suggests a promising path forward to mo
 re resource efficient machine learning may lie in the creation of carefull
 y selected foundation datasets capable of training many different models.\
 n\nBio:\nSurya Ganguli triple majored in physics\, mathematics\, and EECS 
 at MIT\, completed a PhD in string theory at Berkeley\, and a postdoc in t
 heoretical neuroscience at UCSF. He is now an associate professor of Appli
 ed physics at Stanford where he leads the Neural Dynamics and Computation 
 Lab and is a Research Scientist at Meta AI. His research spans the fields 
 of neuroscience\, machine learning and physics\, focusing on understanding
  and improving how both biological and artificial neural networks learn st
 riking emergent computations.  He has been awarded a Swartz-Fellowship in
  computational neuroscience\, a Burroughs-Wellcome Career Award\, a Terman
  Award\, a NeurIPS Outstanding Paper Award\, a Sloan fellowship\, a James 
 S. McDonnell Foundation scholar award in human cognition\, a McKnight Scho
 lar award in Neuroscience\, a Simons Investigator Award in the mathematica
 l modeling of living systems\, and an NSF career award.\n\nThe Center for 
 Intelligent Systems at EPFL (CIS) is a collaboration among IC\, ENAC\, SB\
 ; SV and STI that brings together researchers working on different aspects
  of Intelligent Systems. In June 2020\, CIS has launched its CIS Colloquia
  featuring invited notable speakers.\nMore info\n 
LOCATION:Zoom https://epfl.zoom.us/j/69004498647?pwd=dFhwWFFyaWp4L2VQZllMd
 VFJOGR6dz09 https://epfl.zoom.us/j/69004498647?pwd=dFhwWFFyaWp4L2VQZllMdVF
 JOGR6dz09
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR