BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Theory of Neural Nets Seminar: 31st May 2021
DTSTART:20210531T163000
DTEND:20210531T173000
DTSTAMP:20260506T084813Z
UID:a50ab06e3206f777f639a6dd44f3ef2b30ea729e899f5df910102788
CATEGORIES:Conferences - Seminars
DESCRIPTION:Eran Malach\nThis seminar consists of talks about current rese
 arch on the theory of neural networks. Every session lasts one hour and co
 mprises a talk (about 30 minutes) followed by a discussion with questions 
 from the audience.\n\nSpeaker: Eran Malach (Hebrew University)\n\nTitle:
  On the Benefit of using Differentiable Learning over Tangent Kernels\n\n
 Abstract: A popular line of research in recent years shows that\, in some
  regimes\, optimizing neural networks with gradient descent is equivalent 
 to learning with the neural tangent kernel (NTK) –  a kernel induced by
  the network architecture and initialization. We study the relative power 
 of learning with gradient descent on differentiable models\, such as neura
 l networks\, versus using the corresponding tangent kernels. We show that 
 under certain conditions\, gradient descent achieves small error only if a
  related tangent kernel method achieves a non-trivial advantage over rando
 m guessing (a.k.a. weak learning)\, though this advantage might be very sm
 all even when gradient descent can achieve arbitrarily high accuracy. Comp
 lementing this\, we show that without these conditions\, gradient descent 
 can in fact learn with small error even when no kernel method\, in particu
 lar using the tangent kernel\, can achieve a non-trivial advantage over ra
 ndom guessing.
LOCATION:
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR