BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:External FLAIR seminar : Theodor Misiakiewicz
DTSTART;VALUE=DATE-TIME:20220916T131500
DTEND;VALUE=DATE-TIME:20220916T141500
UID:86afff366c160150f82ed497c8d0502e920765d6a7599546e186801d
CATEGORIES:Conferences - Seminars
DESCRIPTION:Theodor Misiakiewicz\n\nTitle: Learning sparse functions with
neural networks\n\nSpeaker: Theodor Misiakiewicz (Stanford University)\n\
nAbstract: Understanding deep learning requires to understand three compon
ents: approximation (number of parameters to approximate a target function
)\, generalization (number of samples to generalize to unseen data) and co
mputation (typically gradient-based optimization\, number of iterations).
However\, studying their interplay remains a formidable challenge and led
to the introduction of many new ideas (implicit regularization\, tractabil
ity via overparametrization\, benign overfitting etc.).\n\nThis talk will
focus on the setting of learning sparse functions (a function that depends
on a latent low-dimensional subspace) on the hypersphere or hypercube. I
will consider three scenarios corresponding to three optimization regimes
of neural networks (NNs): 1) kernel and random feature regression\; 2) con
vex NNs\; and 3) online SGD on 2-layer NNs in the mean-field scaling. In e
ach of these scenarios\, we provide tight characterizations for each of th
e approximation\, generalization and computational aspects. In particular\
, while NNs trained beyond the kernel regime can adapt to sparsity\, compu
tational aspects cannot be ignored. Understanding which sparse functions a
re efficiently learned by NNs reveals interesting hierarchical structures
in the target function (staircase property) and rich behavior in the SGD d
ynamics (saddles).\n\nThis is based on a few joint works with Emmanuel Abb
e\, Enric Boix-Adsera\, Michael Celentano\, Behrooz Ghorbani\, Hong Hu\, Y
ue M. Lu\, Song Mei and Andrea Montanari. \n
LOCATION:GA 3 21 https://plan.epfl.ch/?room==GA%203%2021
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR