IC Colloquium: Physical Agents that Learn from Experience
Par : Andrew Wagenmaker - UC Berkeley
IC Faculty candidate
Abstract
While the last decade has witnessed tremendous progress in the development of AI, this progress has been concentrated largely in “virtual” realms, and successful applications of AI to the physical world remain limited. Closing this gap and enabling AI in the physical world presents a fundamental challenge: the source of large-scale data that enabled AI progress in the virtual world—the internet—simply does not exist for the physical world.
In this talk, I will investigate how we can overcome this lack of large-scale data by enabling AI agents to learn from experience as they interact with the physical world. In particular, I will consider how we can collect the experience—explore—that allows for learning and improvement, and how the limited sources of data that are often available to us in the physical world—human demonstrations and dynamics models—can enable this. Focusing on robotic control, I will first show how generative robot policies trained on human demonstrations can be utilized to achieve highly focused exploration and enable fast online improvement, and how we can pretrain generative policies on human demonstrations that can themselves collect the experience necessary to learn and improve. I will then consider how dynamics models, even coarse models that are insufficient for obtaining effective task-solving policies, can enable efficient exploration, and how the resulting exploration allows for learning performant task-solving robotic behaviors. Across these examples, I will argue that the insights gained through rigorous analysis are key to uncovering the algorithmic approaches that enable learning from experience, and ultimately bringing AI to the physical world.
Bio
Andrew Wagenmaker is a postdoctoral scholar in Electrical Engineering and Computer Sciences at UC Berkeley working with Sergey Levine. Previously, he completed a PhD in Computer Science at the University of Washington, where he was advised by Kevin Jamieson. Andrew’s research focuses on learning in dynamic, interactive settings, spanning fundamental algorithm development to practical approaches for real-world learning and decision-making, particularly toward enabling efficient learning in robotic systems. His work has been recognized by a Best Paper nomination at the Conference on Robot Learning, and he is a recipient of the NSF Graduate Research Fellowship.
More information
IC Faculty candidate
Abstract
While the last decade has witnessed tremendous progress in the development of AI, this progress has been concentrated largely in “virtual” realms, and successful applications of AI to the physical world remain limited. Closing this gap and enabling AI in the physical world presents a fundamental challenge: the source of large-scale data that enabled AI progress in the virtual world—the internet—simply does not exist for the physical world.
In this talk, I will investigate how we can overcome this lack of large-scale data by enabling AI agents to learn from experience as they interact with the physical world. In particular, I will consider how we can collect the experience—explore—that allows for learning and improvement, and how the limited sources of data that are often available to us in the physical world—human demonstrations and dynamics models—can enable this. Focusing on robotic control, I will first show how generative robot policies trained on human demonstrations can be utilized to achieve highly focused exploration and enable fast online improvement, and how we can pretrain generative policies on human demonstrations that can themselves collect the experience necessary to learn and improve. I will then consider how dynamics models, even coarse models that are insufficient for obtaining effective task-solving policies, can enable efficient exploration, and how the resulting exploration allows for learning performant task-solving robotic behaviors. Across these examples, I will argue that the insights gained through rigorous analysis are key to uncovering the algorithmic approaches that enable learning from experience, and ultimately bringing AI to the physical world.
Bio
Andrew Wagenmaker is a postdoctoral scholar in Electrical Engineering and Computer Sciences at UC Berkeley working with Sergey Levine. Previously, he completed a PhD in Computer Science at the University of Washington, where he was advised by Kevin Jamieson. Andrew’s research focuses on learning in dynamic, interactive settings, spanning fundamental algorithm development to practical approaches for real-world learning and decision-making, particularly toward enabling efficient learning in robotic systems. His work has been recognized by a Best Paper nomination at the Conference on Robot Learning, and he is a recipient of the NSF Graduate Research Fellowship.
More information
Practical information
- General public
- Free
Contact
- Host: Amir Zamir