Deep Harmony: Balancing Utilization and Generality in the Era of Neural Innovation
Event details
| Date | 30.01.2019 |
| Hour | 10:00 › 11:00 |
| Speaker | Hadi Esmaeilzadeh, Professor at University of California, San Diego, USA |
| Location | |
| Category | Conferences - Seminars |
Abstract: Generative Adversarial Networks (GANs) are one of the most recent deep learning models that generate synthetic data from limited genuine datasets. GANs are on the frontier as further extension of deep learning into many domains (e.g., medicine, robotics, content synthesis) requires massive sets of labeled data that is generally either unavailable or prohibitively costly to collect. Although GANs are gaining prominence in various fields, there were no accelerators for these new models. In fact, GANs leverage a new operator, called transposed convolution, that exposes unique challenges for hardware acceleration. This operator first inserts zeros within the multidimensional input, then convolves a kernel over this expanded array to add information to the embedded zeros. Even though there is a convolution stage in this operator, the inserted zeros lead to underutilization of the compute resources when a conventional convolution accelerator is employed. This talk presents the GANAX architecture to alleviate the sources of inefficiency associated with the acceleration of GANs using conventional convolution accelerators, making the first GAN accelerator design possible. We propose a reorganization of the output computations to allocate compute rows with similar patterns of zeros to adjacent processing engines, which also avoids inconsequential multiply-adds on the zeros. The talk then focuses on how to reduce underutilization using runtime information, Specifically, in numerous modern deep model, the outputs of compute-heavy convolution operations are fed to activation units that output zero if their input is negative. By exploiting this unique algorithmic property, we propose a predictive early activation technique, dubbed SnaPEA. This technique cuts the computation of convolution operations short if it determines that the output will be negative. SnaPEA can operate in two distinct modes, exact and predictive. In the exact mode, with no loss in classification accuracy, SnaPEA statically re-orders the weights based on their signs and periodically performs a single-bit sign check on the partial sum. The encouraging results from these innovations how balanced designs pave the way for accelerating the next generation of deep neural models.
Bio: Dr. Esmaeilzadeh was awarded early tenure at the University of California, San Diego (UCSD), where he is the inaugural holder of Halicioglu Chair in Computer Architecture with the rank of associate professor in Computer Science and Engineering. Prior to UCSD, he was an assistant professor in the School of Computer Science at the Georgia Institute of Technology from 2013 to 2017. There, he was the inaugural holder of the Catherine M. and James E. Allchin Early Career Professorship. Hadi is the founding director of the Alternative Computing Technologies (ACT) Lab, where his team is developing new technologies and cross-stack solutions to build the next generation computer systems. He is also the associate director of Center for Machine Integrated Compu=ng and Security (MICS) at UCSD. Dr. Esmaeilzadeh obtained his Ph.D. from the Department of Computer Science and Engineering at the University of Washington in 2013 where his Ph.D. work received the 2013 William Chan Memorial Best Dissertation Award. Prof. Esmaeilzadeh received the IEEE Technical Committee on Computer Architecture (TCCA) “Young Architect” Award in 2018 and was inducted to the ISCA Hall of Fame in the same year. He has received the Air Force Office of Scientific Research Young Investigator Award (2017), College of Computing Outstanding Junior Faculty Research Award (2017), Qualcomm Research Award (2017 and 2016), Google Research Faculty Award (2016 and 2014), Microsoft Research Award (2017 and 2016), and Lockheed Inspirational Young Faculty Award (2016). His teams were awarded the Qualcomm Innovation Fellowship in 2014 and 2018, one of his students was a Microsoft Research Fellow, and another won the 2017 National Center for Women & IT (NCWIT) Collegiate Award. Four of his undergraduate students have been awarded the Georgia Tech President’s Undergraduate Research Award (PURA). His research has been recognized by four Communications of the ACM Research Highlights, four IEEE Micro Top Picks, a nomination for Communications of the ACM Research Highlights, an honorable mention in IEEE Micro Top Picks, and a Distinguished Paper Award in HPCA 2016. Hadi’s work on dark silicon has also been profiled in New York Times. More information is available on his webpage, http://cseweb.ucsd.edu/~hadi/.
Bio: Dr. Esmaeilzadeh was awarded early tenure at the University of California, San Diego (UCSD), where he is the inaugural holder of Halicioglu Chair in Computer Architecture with the rank of associate professor in Computer Science and Engineering. Prior to UCSD, he was an assistant professor in the School of Computer Science at the Georgia Institute of Technology from 2013 to 2017. There, he was the inaugural holder of the Catherine M. and James E. Allchin Early Career Professorship. Hadi is the founding director of the Alternative Computing Technologies (ACT) Lab, where his team is developing new technologies and cross-stack solutions to build the next generation computer systems. He is also the associate director of Center for Machine Integrated Compu=ng and Security (MICS) at UCSD. Dr. Esmaeilzadeh obtained his Ph.D. from the Department of Computer Science and Engineering at the University of Washington in 2013 where his Ph.D. work received the 2013 William Chan Memorial Best Dissertation Award. Prof. Esmaeilzadeh received the IEEE Technical Committee on Computer Architecture (TCCA) “Young Architect” Award in 2018 and was inducted to the ISCA Hall of Fame in the same year. He has received the Air Force Office of Scientific Research Young Investigator Award (2017), College of Computing Outstanding Junior Faculty Research Award (2017), Qualcomm Research Award (2017 and 2016), Google Research Faculty Award (2016 and 2014), Microsoft Research Award (2017 and 2016), and Lockheed Inspirational Young Faculty Award (2016). His teams were awarded the Qualcomm Innovation Fellowship in 2014 and 2018, one of his students was a Microsoft Research Fellow, and another won the 2017 National Center for Women & IT (NCWIT) Collegiate Award. Four of his undergraduate students have been awarded the Georgia Tech President’s Undergraduate Research Award (PURA). His research has been recognized by four Communications of the ACM Research Highlights, four IEEE Micro Top Picks, a nomination for Communications of the ACM Research Highlights, an honorable mention in IEEE Micro Top Picks, and a Distinguished Paper Award in HPCA 2016. Hadi’s work on dark silicon has also been profiled in New York Times. More information is available on his webpage, http://cseweb.ucsd.edu/~hadi/.
Practical information
- Informed public
- Free
Organizer
- Babak Falsafi
Contact
- Stéphanie Baillargues