BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:External FLAIR seminar : Theodor Misiakiewicz
DTSTART:20220916T131500
DTEND:20220916T141500
DTSTAMP:20260509T235631Z
UID:86afff366c160150f82ed497c8d0502e920765d6a7599546e186801d
CATEGORIES:Conferences - Seminars
DESCRIPTION:Theodor Misiakiewicz\n\nTitle: Learning sparse functions with 
 neural networks\n\nSpeaker: Theodor Misiakiewicz (Stanford University)\n\
 nAbstract: Understanding deep learning requires to understand three compon
 ents: approximation (number of parameters to approximate a target function
 )\, generalization (number of samples to generalize to unseen data) and co
 mputation (typically gradient-based optimization\, number of iterations). 
 However\, studying their interplay remains a formidable challenge and led 
 to the introduction of many new ideas (implicit regularization\, tractabil
 ity via overparametrization\, benign overfitting etc.).\n\nThis talk will 
 focus on the setting of learning sparse functions (a function that depends
  on a latent low-dimensional subspace) on the hypersphere or hypercube. I 
 will consider three scenarios corresponding to three optimization regimes 
 of neural networks (NNs): 1) kernel and random feature regression\; 2) con
 vex NNs\; and 3) online SGD on 2-layer NNs in the mean-field scaling. In e
 ach of these scenarios\, we provide tight characterizations for each of th
 e approximation\, generalization and computational aspects. In particular\
 , while NNs trained beyond the kernel regime can adapt to sparsity\, compu
 tational aspects cannot be ignored. Understanding which sparse functions a
 re efficiently learned by NNs reveals interesting hierarchical structures 
 in the target function (staircase property) and rich behavior in the SGD d
 ynamics (saddles).\n\nThis is based on a few joint works with Emmanuel Abb
 e\, Enric Boix-Adsera\, Michael Celentano\, Behrooz Ghorbani\, Hong Hu\, Y
 ue M. Lu\, Song Mei and Andrea Montanari. \n 
LOCATION:GA 3 21 https://plan.epfl.ch/?room==GA%203%2021
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR