BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Hardware-friendly Structured N:M Sparsity for DNN training
DTSTART:20230707T100000
DTEND:20230707T120000
DTSTAMP:20260408T000124Z
UID:230ac78a2985becca5f3a6d2eb8fb1652a8fbede3103e2a96e640d13
CATEGORIES:Conferences - Seminars
DESCRIPTION:Ayan Chakraborty\nEDIC candidacy exam\nExam president: Prof. M
 artin Jaggi\nThesis advisor: Prof. Babak Falsafi\nCo-examiner: Prof. Anne-
 Marie Kermarrec\n\nAbstract\nN:M sparsity has emerged as a form of structu
 red sparsity that achieves comparable accuracies at high sparsity levels c
 ompared to unstructured sparsity techniques. N:M sparsity is relatively ef
 ficient to implement in hardware\, and already enjoys micro-architectural 
 support inside recent NVIDIA GPUs. This makes N:M sparsity an attractive c
 hoice of sparsity scheme to implement for DNN training. DNN Training consi
 sts of 3 General Matrix Multiplications (GEMMs) per linear layer per itera
 tion\, which make up the majority of all computations in Training. Hence\,
  it is essential to be able to apply N:M sparsity to each of these 3 GEMMs
  in order to achieve maximum benefits.\n\nBackground papers\n\n	Aojun Zhou
 \, Yukun Ma\, Junnan Zhu\, Jianbo Liu\, Zhijie Zhang\, Kun Yuan\, Wenxiu S
 un\, and Hongsheng Li. Learning N:M fine-grained structured sparse neural 
 networks from scratch. In International Conference on Learning Representat
 ions (ICLR)\, 2021. Link: https://openreview.net/pdf?id=K9bw7vqp_s\n	Itay 
 Hubara\, Brian Chmiel\, Moshe Island\, Ron Banner\, Joseph Naor\, and Dani
 el Soudry. Accelerated sparse\n	neural training: A provable and efficient 
 method to find N:M transposable masks. In Advances in Neural\n	Information
  Processing Systems (NeurIPS)\, 2021\n	Link: https://openreview.net/pdf?id
 =vRWZsBLKqA\n	Brian Chmiel\, Itay Hubara\, Ron Banner\, and Daniel Soudry.
  Minimum variance unbiased N:M sparsity for\n	the neural gradients. In The
  Eleventh International Conference on Learning Representations (ICLR)\, 20
 23\n	Link: https://openreview.net/pdf?id=vuD2xEtxZcj\n
LOCATION:BC 129 https://plan.epfl.ch/?room==BC%20129
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR