EDIC mock interview public talk: Towards robust weight sharing neural architecture search
Event details
| Date | 23.11.2020 |
| Hour | 16:00 › 17:00 |
| Speaker | EDIC PhD candidate, Kaicheng Yu, CVLAB |
| Location |
Zoom
Online
|
| Category | Conferences - Seminars |
abstract:
Neural architecture search (NAS) aims to facilitate the design of deep networks for new tasks, and has drawn an increasing attention in the past few years. Weight sharing approach, that utilizing a super-net to encompass all possible architectures within the search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. However, we find that (i) On average, some popular NAS algorithms perform similarly to the random policy, (ii) this widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We further decouple weight sharing from the NAS sampling policy, and isolates 14 factors of super-net training. To further improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net of the stand-alone architectures using a small set of landmark architectures.
short bio: Kaicheng is a 4th year Ph.D. candidate in the computer vision lab of EPFL, under the supervision of Dr. Mathieu Salzmann and Prof. Pascal Fua. His research is focusing on improving the neural architecture search algorithms that automatically discover good architectures with minimum human effort. He is also interested in optimizing convolutional neural network (CNN) architectures and operations for mobile devices, and deployed on fine-grained classification and ego-centric view of hand segmentation. Kaicheng is a recipient of the Qualcomm Innovation Fellowship (Europe) 2019. He received his Bachelor in Computer Science from the University of Hong Kong with first-class honor in 2016.
Neural architecture search (NAS) aims to facilitate the design of deep networks for new tasks, and has drawn an increasing attention in the past few years. Weight sharing approach, that utilizing a super-net to encompass all possible architectures within the search space, has become a de facto standard in NAS because it enables the search to be done on commodity hardware. However, we find that (i) On average, some popular NAS algorithms perform similarly to the random policy, (ii) this widely-adopted weight sharing strategy degrades the ranking of the NAS candidates to the point of not reflecting their true performance, thus reducing the effectiveness of the search process. We further decouple weight sharing from the NAS sampling policy, and isolates 14 factors of super-net training. To further improve the super-net quality, we propose a regularization term that aims to maximize the correlation between the performance rankings of the super-net of the stand-alone architectures using a small set of landmark architectures.
short bio: Kaicheng is a 4th year Ph.D. candidate in the computer vision lab of EPFL, under the supervision of Dr. Mathieu Salzmann and Prof. Pascal Fua. His research is focusing on improving the neural architecture search algorithms that automatically discover good architectures with minimum human effort. He is also interested in optimizing convolutional neural network (CNN) architectures and operations for mobile devices, and deployed on fine-grained classification and ego-centric view of hand segmentation. Kaicheng is a recipient of the Qualcomm Innovation Fellowship (Europe) 2019. He received his Bachelor in Computer Science from the University of Hong Kong with first-class honor in 2016.
Practical information
- General public
- Free