Training and Tuning Deep Neural Networks: Faster, Stronger and Better
Event details
| Date | 23.08.2018 |
| Hour | 09:30 › 11:30 |
| Speaker | Chen Liu |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Pascal Frossard
Thesis advisor: Prof. Volkan Cevher
Co-examiner: Prof. Alexandre Alahi
Abstract
Deep neural networks dominate the state-of-the-art models in tasks of computer vision, speech, bioinformatics etc. However, training modern deep neural networks is difficult, because of the highly non-convexity of the objective functions. On one hand, deep neural networks usually have overwhelmingly large number of parameters, they exhibit inhomogeneous curvatures along different directions. This place barriers for training algorithms to escape saddle points. Numerous local minima in this ultra-high dimension space also poses a challenge for training algorithms to find a local minima which has good generalization property. On the other hand, most modern neural network network models are vulnerable to some well-designed perturbation of the input. For most cases, very small and even imperceptible perturbations of input data can fool some sophisticated models. How to obtain a more robust model triggers more attention to the curvature of training objective with respect to the input space. How to provide some guarantee of the trained model against various input perturbations is also a open research question.
My research will mainly focus on the training objective of deep neural networks in two aspects. The first focus on the parameter space to design faster optimizers to find local minima with good generalization property faster. The second focus on the input space to find models robust to input perturbations.
Background papers
Towards deep learning models resistant to adversarial attacks, by Madry, A., et al, arXiv preprint arXiv:1706.06083.
Certifying some distributional robustness with principled adversarial training, by Aman, S., et al, (2018).
Provable defenses against adversarial examples via the convex outer adversarial polytope, by Kolter J Z, Wong E , arXiv preprint arXiv:1711.00851, 2017.
Exam president: Prof. Pascal Frossard
Thesis advisor: Prof. Volkan Cevher
Co-examiner: Prof. Alexandre Alahi
Abstract
Deep neural networks dominate the state-of-the-art models in tasks of computer vision, speech, bioinformatics etc. However, training modern deep neural networks is difficult, because of the highly non-convexity of the objective functions. On one hand, deep neural networks usually have overwhelmingly large number of parameters, they exhibit inhomogeneous curvatures along different directions. This place barriers for training algorithms to escape saddle points. Numerous local minima in this ultra-high dimension space also poses a challenge for training algorithms to find a local minima which has good generalization property. On the other hand, most modern neural network network models are vulnerable to some well-designed perturbation of the input. For most cases, very small and even imperceptible perturbations of input data can fool some sophisticated models. How to obtain a more robust model triggers more attention to the curvature of training objective with respect to the input space. How to provide some guarantee of the trained model against various input perturbations is also a open research question.
My research will mainly focus on the training objective of deep neural networks in two aspects. The first focus on the parameter space to design faster optimizers to find local minima with good generalization property faster. The second focus on the input space to find models robust to input perturbations.
Background papers
Towards deep learning models resistant to adversarial attacks, by Madry, A., et al, arXiv preprint arXiv:1706.06083.
Certifying some distributional robustness with principled adversarial training, by Aman, S., et al, (2018).
Provable defenses against adversarial examples via the convex outer adversarial polytope, by Kolter J Z, Wong E , arXiv preprint arXiv:1711.00851, 2017.
Practical information
- General public
- Free
Contact
- EDIC - [email protected]