IC Colloquium: Green AI
By: Roy Schwartz - Hebrew University of Jerusalem
Video of his talk
Abstract
The computations required for deep learning research have been doubling every few months, resulting in an estimated 5,000x increase from 2018 to 2022. This trend has led to unprecedented success in a range of AI tasks. In this talk I will discuss a few troubling side-effects of this trend, touching on issues of lack of inclusiveness within the research community, and an increasingly large environmental footprint.
I will then present Green AI – an alternative approach to help mitigate these concerns. Green AI is composed of two main ideas: increased reporting of computational budgets, and making efficiency an evaluation criterion for research alongside accuracy and related measures. I will focus on the latter topic, discussing various recent approaches for reducing the computational cost of AI, some based on sample efficiency, others on more efficient mathematical modeling, and also on methods for adaptive computation.
This is joint work with André F. T. Martins, Betty van Aken, Colin Raffel, Dani Yogatama, Daniel Rotem, Edwin Simpson, Emma Strubell, Erika Odmark, Gabriel Stanovsky, Hannaneh Hajishirzi, Hao Peng, Ivan Montero, Jesse Dodge, Ji-Ung Lee, Jungo Kasai, Kenneth Heafield, Leon Derczynski, Lingpeng Kong, Manuel R. Ciosici, Marcos Treviso, Michael Elhadad, Michael Hassid, Nicholas Lourie, Nicole DeCario, Nikolaos Pappas, Niranjan Balasubramanian, Noah A. Smith, Noam Slonim, Oren Etzioni, Pedro H. Martins, Peter Milder, Qingqing Cao, Remi Tachet des Combes, Sara Hooker, Sasha Luccioni, Swabha Swayamdipta, Taylor Prewitt, Tianchu Ji, Will Buchanan, Yejin Choi, Yizhong Wang, Yonatan Bitton and Zhaofeng Wu.
Bio
Roy Schwartz is a senior lecturer (assistant professor) at the School of Computer Science and Engineering at The Hebrew University of Jerusalem (HUJI). Roy studies natural language processing and artificial intelligence. Prior to joining HUJI, Roy was a postdoc (2016-2019) and then a research scientist (2019-2020) at the Allen institute for AI and at the School of Computer Science and Engineering at The University of Washington, where he worked with Noah A. Smith. Roy completed his Ph.D. in 2016 at the School of Computer Science and Engineering at HUJI, where he worked with Ari Rappoport. Roy’s work has appeared on the cover of the CACM magazine, and has been featured, among others, in the New York Times, MIT Tech Review, and Forbes.
More information
Video of his talk
Abstract
The computations required for deep learning research have been doubling every few months, resulting in an estimated 5,000x increase from 2018 to 2022. This trend has led to unprecedented success in a range of AI tasks. In this talk I will discuss a few troubling side-effects of this trend, touching on issues of lack of inclusiveness within the research community, and an increasingly large environmental footprint.
I will then present Green AI – an alternative approach to help mitigate these concerns. Green AI is composed of two main ideas: increased reporting of computational budgets, and making efficiency an evaluation criterion for research alongside accuracy and related measures. I will focus on the latter topic, discussing various recent approaches for reducing the computational cost of AI, some based on sample efficiency, others on more efficient mathematical modeling, and also on methods for adaptive computation.
This is joint work with André F. T. Martins, Betty van Aken, Colin Raffel, Dani Yogatama, Daniel Rotem, Edwin Simpson, Emma Strubell, Erika Odmark, Gabriel Stanovsky, Hannaneh Hajishirzi, Hao Peng, Ivan Montero, Jesse Dodge, Ji-Ung Lee, Jungo Kasai, Kenneth Heafield, Leon Derczynski, Lingpeng Kong, Manuel R. Ciosici, Marcos Treviso, Michael Elhadad, Michael Hassid, Nicholas Lourie, Nicole DeCario, Nikolaos Pappas, Niranjan Balasubramanian, Noah A. Smith, Noam Slonim, Oren Etzioni, Pedro H. Martins, Peter Milder, Qingqing Cao, Remi Tachet des Combes, Sara Hooker, Sasha Luccioni, Swabha Swayamdipta, Taylor Prewitt, Tianchu Ji, Will Buchanan, Yejin Choi, Yizhong Wang, Yonatan Bitton and Zhaofeng Wu.
Bio
Roy Schwartz is a senior lecturer (assistant professor) at the School of Computer Science and Engineering at The Hebrew University of Jerusalem (HUJI). Roy studies natural language processing and artificial intelligence. Prior to joining HUJI, Roy was a postdoc (2016-2019) and then a research scientist (2019-2020) at the Allen institute for AI and at the School of Computer Science and Engineering at The University of Washington, where he worked with Noah A. Smith. Roy completed his Ph.D. in 2016 at the School of Computer Science and Engineering at HUJI, where he worked with Ari Rappoport. Roy’s work has appeared on the cover of the CACM magazine, and has been featured, among others, in the New York Times, MIT Tech Review, and Forbes.
More information
Practical information
- General public
- Free
- This event is internal
Contact
- Host: Nicolas Flammarion