BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Scaling Distributed Deep Learning with Efficient Algorithm Design
DTSTART:20180613T100000
DTEND:20180613T120000
DTSTAMP:20260407T060503Z
UID:33cfb83065f457f5843393521ce6d659bf556ec2e2c0edc6022e32ad
CATEGORIES:Conferences - Seminars
DESCRIPTION:Tao LIN\nEDIC candidacy exam\nExam president: Prof. François 
 Fleuret\nThesis advisor: Prof. Martin Jaggi\nThesis co-advisor: Prof. Baba
 k Falsafi\nCo-examiner: Prof. Rachid Guerraoui\n\nAbstract\nDue to the rap
 id growth of data and the ever-increasing model complexity\, today\, most 
 important deep learning algorithms cannot be efficiently solved by a singl
 e machine. Distributed training architectures for training have been devel
 oped in response to the challenges\, and promise improved scalability by i
 ncreasing both computational and storage capacities. A critical challenge 
 in realizing this promise of scalability is to develop efficient methods f
 or communicating and coordinating information between distributed devices\
 , taking into account the specific needs of machine learning training algo
 rithms. On most distributed systems\, the communication of information bet
 ween devices is vastly more expensive than reading data from main memory a
 nd performing the local computation. Moreover\, the optimal trade-off betw
 een communication and computation can vary widely depending on the dataset
  being processed\, the available system resources being used\, and the tra
 ining objective being optimized. In this thesis\, we try to address the ab
 ove-mentioned challenge\, for the improvement of scalability of learning s
 ystems.\n\nBackground papers\nQSGD: Communication-Efficient SGD via Gradie
 nt Quantization and Encoding\, by Alistarh\, Dan\, et al.  Advances in Ne
 ural Information Processing Systems. 2017.\nDeep Gradient Compression: Red
 ucing the Communication Bandwidth for Distributed Training\, by Lin\, Yuju
 n\, et al. arXiv preprint arXiv:1712.01887 (2017).​ \nAccurate\, large 
 minibatch SGD: training imagenet in 1 hour\, by Goyal\, Priya\, et al. arX
 iv preprint arXiv:1706.02677 (2017).​ 
LOCATION:INJ 328 https://plan.epfl.ch/?room=INJ328
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR