Addressing Computational and Statistical Gaps with Deep Neural Networks

Event details
Date | 07.10.2016 |
Hour | 11:15 › 12:30 |
Speaker | Joan BRUNA |
Location | |
Category | Conferences - Seminars |
Many modern statistical questions are plagued with asymptotic regimes that separate our current theoretical understanding with what is possible given finite computational and sample resources. Important examples of such gaps appear in sparse inference, high-dimensional density estimation and non-convex optimization.
In the former, proximal splitting algorithms efficiently solve the l1-relaxed sparse coding problem, but their performance is typically evaluated in terms of asymptotic convergence rates. In unsupervised high-dimensional learning, a major challenge is how to appropriately combine prior knowledge in order to beat the curse of dimensionality. Finally, the prevailing dichotomy between convex and non-convex optimization is not adapted to describe the diversity
of optimization scenarios faced as soon as convexity fails. In this talk we will illustrate how Deep architectures can be used in order to attack such gaps. We will first see how a neural network sparse coding model (LISTA, Gregor & LeCun’10) can be analyzed in terms of a particular matrix factorization of the
dictionary, which leverages diagonalisation with invariance of the l1 ball, revealing a phase transition that is consistent with numerical experiments. We will then discuss image and texture generative modeling and super-resolution, a prime example of high-dimensional inverse problem. In that setting, we will
explain how multi-scale convolutional neural networks are equipped to beat the curse of dimensionality and provide stable estimation of high frequency information. Finally, we will discuss recent research in which we explore to what extent the non-convexity of the loss surface arising in deep learning problems is hurting gradient descent algorithms, by efficiently estimating the number of basins of attractions.
Bio: Joan graduated cum-laude from Universitat Politècnica de Catalunya in both Mathematics and Telecommunications Engineering, before graduating in Applied Mathematics from ENS Cachan (France). He then became a Sr. Research Engineer in an Image Processing startup, developing real-time video processing algorithms. He obtained his PhD in Applied Mathematics at École Polytechnique (France). After a postdoctoral stay at Courant Institute, NYU, he became a Postdoctoral fellow at Facebook AI Research. He is an Assistant Professor at UC Berkeley, Statistics Department (on leave), and since Fall 2016 he is Assistant Professor at Courant Institute, NYU (Computer Science, Center for Data Science and Mathematics (courtesy)).
His research interests include invariant signal representations, deep learning, high-dimensional statistics, and its applications to computer vision, statistical physics and AI.
In the former, proximal splitting algorithms efficiently solve the l1-relaxed sparse coding problem, but their performance is typically evaluated in terms of asymptotic convergence rates. In unsupervised high-dimensional learning, a major challenge is how to appropriately combine prior knowledge in order to beat the curse of dimensionality. Finally, the prevailing dichotomy between convex and non-convex optimization is not adapted to describe the diversity
of optimization scenarios faced as soon as convexity fails. In this talk we will illustrate how Deep architectures can be used in order to attack such gaps. We will first see how a neural network sparse coding model (LISTA, Gregor & LeCun’10) can be analyzed in terms of a particular matrix factorization of the
dictionary, which leverages diagonalisation with invariance of the l1 ball, revealing a phase transition that is consistent with numerical experiments. We will then discuss image and texture generative modeling and super-resolution, a prime example of high-dimensional inverse problem. In that setting, we will
explain how multi-scale convolutional neural networks are equipped to beat the curse of dimensionality and provide stable estimation of high frequency information. Finally, we will discuss recent research in which we explore to what extent the non-convexity of the loss surface arising in deep learning problems is hurting gradient descent algorithms, by efficiently estimating the number of basins of attractions.
Bio: Joan graduated cum-laude from Universitat Politècnica de Catalunya in both Mathematics and Telecommunications Engineering, before graduating in Applied Mathematics from ENS Cachan (France). He then became a Sr. Research Engineer in an Image Processing startup, developing real-time video processing algorithms. He obtained his PhD in Applied Mathematics at École Polytechnique (France). After a postdoctoral stay at Courant Institute, NYU, he became a Postdoctoral fellow at Facebook AI Research. He is an Assistant Professor at UC Berkeley, Statistics Department (on leave), and since Fall 2016 he is Assistant Professor at Courant Institute, NYU (Computer Science, Center for Data Science and Mathematics (courtesy)).
His research interests include invariant signal representations, deep learning, high-dimensional statistics, and its applications to computer vision, statistical physics and AI.
Links
Practical information
- Informed public
- Free
- This event is internal
Organizer
- Pascal FROSSARD
Contact
- Pascal FROSSARD