BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Theory of Neural Nets Seminar: 5th July 2021
DTSTART:20210705T163000
DTEND:20210705T173000
DTSTAMP:20260509T161101Z
UID:8aac13dade28af864e3bdf60437999e7508fb69eb7ffbecd9a3c0c25
CATEGORIES:Conferences - Seminars
DESCRIPTION:This seminar consists of talks about current research on the t
 heory of neural networks. Every session lasts one hour and comprises a tal
 k (about 30 minutes) followed by a discussion with questions from the audi
 ence.\n\nSpeaker: Matus Telgarsky (University of Illinois)\n\nTitle: Log
 istic regression explicitly maximizes margins\; should we stop training ea
 rly?\n\nAbstract: This talk will present two perspectives on the behavior
  of gradient descent with the logistic loss: on the one hand\, it seems we
  should run as long as possible\, and achieve good margins\; on the other\
 , stopping early seems necessary for noisy problems.  In the first part\,
  focused on the linear case\, a new perspective of explicit bias (rather t
 han implicit bias) yields new analyses and algorithms with margin maximiza
 tion rate as fast as 1/t^2 (whereas prior work had 1/sqrt{t} at best).  T
 he second part\, focused on shallow ReLU networks\, argues that the margin
  bias might fail to be ideal\, and that stopping early can achieve consist
 ency and calibration for arbitrary classification problems.  Moreover\, t
 his early phase is still adaptive to data simplicity\, but with a differen
 t bias than the margin bias.\n\nJoint work with Ziwei Ji\, Justin D. Li\, 
 Nati Srebro.
LOCATION:
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR