Combining Tile-Wise Sparsity with Quantization for DNN Hardware Accelerators
Event details
| Date | 27.07.2022 |
| Hour | 10:00 › 12:00 |
| Speaker | Fatih Yazici |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Paolo Ienne
Thesis advisor: Prof. Babak Falsafi
Co-examiner: Prof. Martin Jaggi
Abstract
Deep Neural Networks (DNNs) demonstrated superior accuracy over conventional methods in numerous problems including but not limited to image recognition, speech recognition, text translation, and self-driving cars. However, DNNs are comprised of billions of parameters, which renders the task of training them a challenge given the traditional computing platforms like CPUs. In this proposal, we compare various approaches to DNN hardware accelerators. Dense hardware accelerators offer high throughput but fail to capture the redundant nature of DNNs. Sparse hardware accelerator solutions utilize this redundancy but face challenges in implementation due to increased architectural complexity. Software solutions promise a good trade-off to use sparsity while executing on efficient dense hardware.
Background papers
[1] N. P. Jouppi et al., "Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product," 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 2021, pp. 1-14, doi: 10.1109/ISCA52012.2021.00010.
Available on: https://ieeexplore.ieee.org/document/9499913
[2] E. Qin et al., "SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training," 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2020, pp. 58-70, doi: 10.1109/HPCA47549.2020.00015.
Available on: https://ieeexplore.ieee.org/document/9065523
[3] C. Guo et al., "Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsity," SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 2020, pp. 1-15, doi: 10.1109/SC41405.2020.00020.
Available on: https://ieeexplore.ieee.org/document/9355304
Exam president: Prof. Paolo Ienne
Thesis advisor: Prof. Babak Falsafi
Co-examiner: Prof. Martin Jaggi
Abstract
Deep Neural Networks (DNNs) demonstrated superior accuracy over conventional methods in numerous problems including but not limited to image recognition, speech recognition, text translation, and self-driving cars. However, DNNs are comprised of billions of parameters, which renders the task of training them a challenge given the traditional computing platforms like CPUs. In this proposal, we compare various approaches to DNN hardware accelerators. Dense hardware accelerators offer high throughput but fail to capture the redundant nature of DNNs. Sparse hardware accelerator solutions utilize this redundancy but face challenges in implementation due to increased architectural complexity. Software solutions promise a good trade-off to use sparsity while executing on efficient dense hardware.
Background papers
[1] N. P. Jouppi et al., "Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product," 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 2021, pp. 1-14, doi: 10.1109/ISCA52012.2021.00010.
Available on: https://ieeexplore.ieee.org/document/9499913
[2] E. Qin et al., "SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training," 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2020, pp. 58-70, doi: 10.1109/HPCA47549.2020.00015.
Available on: https://ieeexplore.ieee.org/document/9065523
[3] C. Guo et al., "Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsity," SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 2020, pp. 1-15, doi: 10.1109/SC41405.2020.00020.
Available on: https://ieeexplore.ieee.org/document/9355304
Practical information
- General public
- Free