Learning beyond Classical Generalization with Neural Networks
Event details
| Date | 28.07.2022 |
| Hour | 15:00 › 17:00 |
| Speaker | Aryo Lotfi Jandaghi |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Martin Jaggi
Thesis advisor: Prof. Emmanuel Abbé
Co-examiner: Prof. Lénaïc Chizat
Abstract
In this report, we focus on the generalization ability
of neural networks. We describe the recent trend of analyzing
neural networks’ reasoning capability in contrast to their memorization ability.
We reintroduce the framework of Boolean pointer
value retrieval [1] and explain how it can help us understand the
mechanisms behind the generalization of neural networks. We
present three recent papers on: (i) the spectral bias of neural
networks, (ii) learning sparse Boolean functions in the meanfield
regime, and (iii) convergence of deep linear neural networks.
Finally, we explain how these papers can guide our research and
describe our short and long term research plans.
Background papers
Exam president: Prof. Martin Jaggi
Thesis advisor: Prof. Emmanuel Abbé
Co-examiner: Prof. Lénaïc Chizat
Abstract
In this report, we focus on the generalization ability
of neural networks. We describe the recent trend of analyzing
neural networks’ reasoning capability in contrast to their memorization ability.
We reintroduce the framework of Boolean pointer
value retrieval [1] and explain how it can help us understand the
mechanisms behind the generalization of neural networks. We
present three recent papers on: (i) the spectral bias of neural
networks, (ii) learning sparse Boolean functions in the meanfield
regime, and (iii) convergence of deep linear neural networks.
Finally, we explain how these papers can guide our research and
describe our short and long term research plans.
Background papers
- On the Spectral Bias of Neural Networks (link: https://proceedings.mlr.press/v97/rahaman19a/rahaman19a.pdf)
- The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks (link: https://proceedings.mlr.press/v178/abbe22a/abbe22a.pdf)
- A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks (link: https://openreview.net/pdf?id=SkMQg3C5K7)
Practical information
- General public
- Free