Data-Efficient Learning in Autonomous Robots
Event details
| Date | 22.04.2016 |
| Hour | 14:15 › 15:15 |
| Speaker |
Marc Deisenroth, ICL Bio: Marc Deisenroth is a Lecturer in Statistical Machine Learning at the Department of Computing, Imperial College London. Prior to his appointment, he was an Imperial College Research Fellow (09/2013–06/2015), Senior Research Scientist & Group Leader at TU Darmstadt (12/2011–08/2013), and Research Associate at the University of Washington and Intel Labs Seattle (02/2010–12/2011). Marc completed his PhD in 2009 with Carl Edward Rasmussen. Marc was Program Chair of EWRL 2012 and received a Best Paper Award at ICRA 2014. He is a recipient of a Google Faculty Research Award and a Microsoft PhD Scholarship. Marc’s research interests center around data-efficient machine learning methods (with a focus on Bayesian methods), with the objective to increase the level of autonomy in learning systems by modeling and accounting for uncertainty in a principled way. Potential applications include personalized healthcare, autonomous robots and bio-chemical systems. |
| Location | |
| Category | Conferences - Seminars |
Fully autonomous systems and robots have been a vision for many decades, but we are still far from practical realization. One of the fundamental challenges in fully autonomous systems and robots is learning from data directly without relying on any kind of intricate human knowledge. This requires data-driven statistical methods for modeling, predicting, and decision making, while taking uncertainty into account, e.g., due to measurement noise, sparse data or stochasticity in the environment.
In my talk I will focus on machine learning methods for controlling autonomous robots, which pose an additional practical challenge: Data-efficiency, i.e., we need to be able to learn controllers in a few experiments since performing millions of experiments with robots is time consuming and wears out the hardware. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, pre-shaped policies, or the underlying dynamics.
In the first part of the talk, I follow a different approach and speed up learning by efficiently extracting information from sparse data. In particular, I propose to learn a probabilistic, non-parametric Gaussian process dynamics model. By explicitly incorporating model uncertainty in long-term planning and controller learning my approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art reinforcement learning our model-based policy search method achieves an unprecedented speed of learning, which makes is most promising for application to real systems. I demonstrate its applicability to autonomous learning from scratch on real robot and control tasks.
In the second part of my talk, I will discuss an alternative method for learning controllers for bipedal locomotion based on Bayesian Optimization, where it is hard to learn models of the underlying dynamics due to ground contacts. Using Bayesian optimization, we sidestep this modeling issue and directly optimize the controller parameters without the need of modeling the robot's dynamics.
In my talk I will focus on machine learning methods for controlling autonomous robots, which pose an additional practical challenge: Data-efficiency, i.e., we need to be able to learn controllers in a few experiments since performing millions of experiments with robots is time consuming and wears out the hardware. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, pre-shaped policies, or the underlying dynamics.
In the first part of the talk, I follow a different approach and speed up learning by efficiently extracting information from sparse data. In particular, I propose to learn a probabilistic, non-parametric Gaussian process dynamics model. By explicitly incorporating model uncertainty in long-term planning and controller learning my approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art reinforcement learning our model-based policy search method achieves an unprecedented speed of learning, which makes is most promising for application to real systems. I demonstrate its applicability to autonomous learning from scratch on real robot and control tasks.
In the second part of my talk, I will discuss an alternative method for learning controllers for bipedal locomotion based on Bayesian Optimization, where it is hard to learn models of the underlying dynamics due to ground contacts. Using Bayesian optimization, we sidestep this modeling issue and directly optimize the controller parameters without the need of modeling the robot's dynamics.
Practical information
- Expert
- Free