IC Colloquium : Optimization, Learning and Systems
Event details
| Date | 15.02.2016 |
| Hour | 10:15 › 11:30 |
| Location | |
| Category | Conferences - Seminars |
By : Martin Jaggi - ETHZ
IC Faculty candidate
Abstract :
The massive growth of available data has moved data analysis and machine learning to center stage in many industrial as well as scientific fields, ranging from astronomy to health science, web and sensor data, and countless other applications. In this talk, we focus on the computational challenges of machine learning on large datasets through the lens of mathematical optimization.
Significant recent research aims to improve the efficiency, scalability, and theoretical understanding of iterative optimization algorithms used for training machine learning models. Motivated by widely used regression and classification techniques, we discuss new results for first-order optimization techniques, empowering them with primal-dual certificates and convergence guarantees which are valuable for practitioners. We also illustrate how Frank-Wolfe and coordinate descent algorithms can help to trade-off accuracy against complexity (measured for example in sparsity or rank) of machine learning models.
While single machine solvers have been highly optimized, they can not easily be transferred to the distributed setting, i.e. when the dataset exceeds the storage capacity of a single computer. For the users of machine learning methods, this lack of generality of learning algorithms is becoming increasingly frustrating, as the complexity and variability of the underlying systems is increasing, for example in terms of differences in communication, computation and memory speeds.
This highlights the necessity for new learning algorithms which are able to efficiently adapt to the available compute environment, spanning a wide range from cheap public cloud computing stacks to more traditional HPC systems. At the same time, the theoretical efficiency guarantees should ideally be adaptive to the real system properties as well. Finally, open source optimization software combined with public benchmarks can help industrial users navigate the increasingly complex landscape of commercial big data software frameworks.
Bio :
Martin Jaggi is a post-doctoral researcher in machine learning at ETH Zurich. Before that, he was a research fellow at the Simons Institute in Berkeley, US, working on the theory of big data analysis, and a post-doctoral researcher at École Polytechnique in Paris, France. He has earned his PhD in Machine Learning and Optimization from ETH Zurich in 2011, and a MSc in Mathematics also from ETH Zurich, interrupted with several shorter stints in industry (Google, Netbreeze, Avaloq). He is broadly interested in methods for the analysis of large datasets, distributed training algorithms, open source software and machine learning applications for example in medicine, computer vision and text analysis. He is a co-founder of the startup SpinningBytes.com, and also the founder and co-organizer of the Zurich Machine Learning and Data Science Meetup, the largest technology Meetup group in Switzerland.
More information
IC Faculty candidate
Abstract :
The massive growth of available data has moved data analysis and machine learning to center stage in many industrial as well as scientific fields, ranging from astronomy to health science, web and sensor data, and countless other applications. In this talk, we focus on the computational challenges of machine learning on large datasets through the lens of mathematical optimization.
Significant recent research aims to improve the efficiency, scalability, and theoretical understanding of iterative optimization algorithms used for training machine learning models. Motivated by widely used regression and classification techniques, we discuss new results for first-order optimization techniques, empowering them with primal-dual certificates and convergence guarantees which are valuable for practitioners. We also illustrate how Frank-Wolfe and coordinate descent algorithms can help to trade-off accuracy against complexity (measured for example in sparsity or rank) of machine learning models.
While single machine solvers have been highly optimized, they can not easily be transferred to the distributed setting, i.e. when the dataset exceeds the storage capacity of a single computer. For the users of machine learning methods, this lack of generality of learning algorithms is becoming increasingly frustrating, as the complexity and variability of the underlying systems is increasing, for example in terms of differences in communication, computation and memory speeds.
This highlights the necessity for new learning algorithms which are able to efficiently adapt to the available compute environment, spanning a wide range from cheap public cloud computing stacks to more traditional HPC systems. At the same time, the theoretical efficiency guarantees should ideally be adaptive to the real system properties as well. Finally, open source optimization software combined with public benchmarks can help industrial users navigate the increasingly complex landscape of commercial big data software frameworks.
Bio :
Martin Jaggi is a post-doctoral researcher in machine learning at ETH Zurich. Before that, he was a research fellow at the Simons Institute in Berkeley, US, working on the theory of big data analysis, and a post-doctoral researcher at École Polytechnique in Paris, France. He has earned his PhD in Machine Learning and Optimization from ETH Zurich in 2011, and a MSc in Mathematics also from ETH Zurich, interrupted with several shorter stints in industry (Google, Netbreeze, Avaloq). He is broadly interested in methods for the analysis of large datasets, distributed training algorithms, open source software and machine learning applications for example in medicine, computer vision and text analysis. He is a co-founder of the startup SpinningBytes.com, and also the founder and co-organizer of the Zurich Machine Learning and Data Science Meetup, the largest technology Meetup group in Switzerland.
More information
Practical information
- General public
- Free
- This event is internal
Contact
- Host : Ola Svensson