BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:IC Colloquium: Memory-Efficient Adaptive Optimization for Humungou
 s-Scale Learning
DTSTART:20191021T161500
DTEND:20191021T173000
DTSTAMP:20260406T214521Z
UID:42152a34595c5dc829575092b2773ba9c7704f09818fa4fb043736ce
CATEGORIES:Conferences - Seminars
DESCRIPTION:By: Yoram Singer - Princeton University\nVideo of his talk\n\n
 Abstract\nAdaptive gradient-based optimizers such as AdaGrad and Adam are 
 among the methods of choice in modern machine learning. These methods main
 tain second-order statistics of each model parameter\, thus doubling the m
 emory footprint of the optimizer. In behemoth-size applications\, this mem
 ory overhead restricts the size of the model being used as well as the num
 ber of examples in a mini-batch. I start by giving a general overview of a
 daptive gradient methods. I then describe a novel\, simple\, and flexible 
 adaptive optimization method with sublinear memory cost that retains the b
 enefits of classical adaptive methods. I give convergence guarantees for t
 he method and demonstrate its effectiveness in training some of the larges
 t deep models.\n\nBio\nYoram Singer is a professor of Computer Science at 
 Princeton University. He was a member of the technical staff at AT&T Resea
 rch 1995-1999\, an associate professor at the Hebrew University 1999-2007\
 , and a Principal Scientist at Google 2005-2019. At Google\, he implemente
 d and launched Google’s Domain Spam classifier used for all search queri
 es 2004-2017\, co-founded the Sibyl system which served YouTube prediction
 s 2008-2018\, founded the Principles Of Effective Machine-learning group\,
  and the Google’s AI Lab at Princeton. He co-chaired COLT’04 and NIPS
 ’04. He is a fellow of AAAI.\n\nMore information
LOCATION:BC 420 https://plan.epfl.ch/?room==BC%20420
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR