BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Gradient-Based Feature Learning under Structured Data
DTSTART:20231020T131500
DTEND:20231020T141500
DTSTAMP:20260407T103133Z
UID:c7920c2bc67a31c953ed78324d0678ab50af251f8f039ee1fe3c88af
CATEGORIES:Conferences - Seminars
DESCRIPTION:Alireza Mousavi-Hosseini (University of Toronto)\nRecent work
 s have demonstrated that in high-dimensional settings\, the sample comple
 xity of gradient-based learning of single index models\, i.e. functions th
 at depend on a 1-dimensional projection of the input data\, is governed by
  a quantity of the model called the information exponent. However\, these
  results are only concerned with isotropic data\, while in practice the in
 put often contains additional structure which can implicitly guide the alg
 orithm. In this talk\, we investigate the effect of a spiked covariance st
 ructure and reveal several interesting phenomena. First\, we show that in 
 the anisotropic setting\, the commonly used spherical gradient dynamics ma
 y fail to recover the true direction\, which can be alleviated by appropri
 ate weight normalization that is reminiscent of batch normalization. Furth
 er\, we demonstrate that depending on the spark-target alignment\, the sam
 ple complexity can go through a three-stage phase transition. In particula
 r\, with a suitably large spike\, the sample complexity of gradient-based 
 training can be made independent of the information exponent while also ou
 tperforming lower bounds for rotationally invariant kernel methods.
LOCATION:GA 3 21 https://plan.epfl.ch/?room==GA%203%2021
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR