BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Gradient Compression Techniques to Accelerate Distributed Training
  of Neural Networks
DTSTART:20190828T103000
DTEND:20190828T123000
DTSTAMP:20260429T133458Z
UID:baa110d29ff19b7ec119ccaf2bb7e95f603a84e0231f5c3d39e50272
CATEGORIES:Conferences - Seminars
DESCRIPTION:Thijs Vogels\nEDIC candidacy exam\nExam president: Prof. Pasca
 l Frossard\nThesis advisor: Prof. Martin Jaggi\nCo-examiner: Prof. Babak F
 alsafi\n\nAbstract\nIn distributed training of machine learning models wit
 h stochastic optimization\, the exchange of parameter updates between work
 ers often is a bottleneck that limits the scalability of distributed train
 ing. This is especially true for models with a large parameter space\, suc
 h as neural networks. Several techniques have been proposed to enhance sca
 lability by compressing gradients\, e.g. by sending a sparse set of coordi
 nates only\, or by quantization. We study the gradient compression literat
 ure from both sides: on the one hand\, we study properties of these algori
 thms in a distributed setting\, and their effectiveness for speed and scal
 ability. On the other hand\, we explore properties of the minima found by 
 these algorithms\, such as robustness or generalisation.\n\nBackground pap
 ers\nQSGD: Communication-Efficient SGD via Gradient Quantization and Encod
 ing\, by Alistarh et al. NIPS 2017.\nATOMO: Communication-efficient Learni
 ng via Atomic Sparsification\, by Wang et al. Neurips 2018.\nError Feedbac
 k Fixes SignSGD and other Gradient Compression Schemes\, by Karimireddy et
  al. ICML 2019. \n 
LOCATION:BC 129 https://plan.epfl.ch/?room==BC%20129
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR