BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Feature learning\, lower-homogeneity\, and normalization layers
DTSTART:20240321T111500
DTEND:20240321T121500
DTSTAMP:20260407T114640Z
UID:5eae918a077d545b296187d69f5e8c408fe155e9752de85e0488c26b
CATEGORIES:Conferences - Seminars
DESCRIPTION:Matus Telgarsky\nThe first half of this talk will describe the
  feature learning problem in deep learning optimization\, its statistical 
 consequences\, and an approach to proving general theorems with a heavy re
 liance on normalization layers\, which are common to all modern architectu
 res but typically treated as an analytic nuisance.  Theorems will cover t
 wo settings: concrete results for shallow networks\, and abstract template
  theorems for general architectures. The shallow network results allow for
  globally maximal margins at the cost of large width and no further assump
 tions\, while the general architecture theorems give convergence rates to 
 KKT points for a new general class of architectures satisfying "partial lo
 wer-homogeneity".\nThe second half will be technical\, demonstrating two c
 ore proof techniques. The first ingredient\, essential to the shallow anal
 ysis\, is a new mirror descent lemma\, strengthening a beautiful idea disc
 overed by Chizat and Bach. The second ingredient is the concept of "partia
 l lower-homogeneity" and its consequences.\n\nJoint work with Danny Son\; 
 not currently on arXiv\, but "coming soon".
LOCATION:GA 3 21 https://plan.epfl.ch/?room==GA%203%2021
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR