Multiplication Free Neural Network Architectures
Event details
| Date | 31.08.2023 |
| Hour | 12:00 › 14:00 |
| Speaker | Bettina Messmer |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. François Fleuret
Thesis advisor: Prof. Martin Jaggi
Co-examiner: Prof. Mathieu Salzmann
Abstract
The remarkable progress of deep learning models
has come with increased energy consumption, impacting the
environment, and limiting deployment on resource constrained
devices, such as mobile devices. Our research goal is to explore
resource-efficient alternatives to state-of-art network architectures,
specifically, we focus on designs that substantially reduce
multiplications. We discuss three significant works: a baseline
architecture notably reducing the multiplicative parameters, a
theoretical model reducing the number of multiplications to a
minimum while retaining universal approximation capabilities,
and a more general exploration of resource efficiency in the
natural language processing (NLP) domain.
Background papers
Exam president: Prof. François Fleuret
Thesis advisor: Prof. Martin Jaggi
Co-examiner: Prof. Mathieu Salzmann
Abstract
The remarkable progress of deep learning models
has come with increased energy consumption, impacting the
environment, and limiting deployment on resource constrained
devices, such as mobile devices. Our research goal is to explore
resource-efficient alternatives to state-of-art network architectures,
specifically, we focus on designs that substantially reduce
multiplications. We discuss three significant works: a baseline
architecture notably reducing the multiplicative parameters, a
theoretical model reducing the number of multiplications to a
minimum while retaining universal approximation capabilities,
and a more general exploration of resource efficiency in the
natural language processing (NLP) domain.
Background papers
Practical information
- General public
- Free