BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Interaction of Neural Architecture and Optimization in Deep Learni
 ng
DTSTART:20220902T130000
DTEND:20220902T150000
DTSTAMP:20260527T163834Z
UID:6e94c03590f178c8336a520285a19febb14a244d1fdc9ee3bf0aa928
CATEGORIES:Conferences - Seminars
DESCRIPTION:Atli Kosson\nEDIC candidacy exam\nExam president: Prof. Amir Z
 amir\nThesis advisor: Prof. Martin Jaggi\nCo-examiner: Prof. François Fle
 uret\n\nAbstract\nModern deep neural networks have a complex structure\, c
 onsisting of many layers as well as different types of trainable parameter
 s such as convolutional filters\, gains and biases. They are predominantly
  optimized using some form of stochastic gradient descent (SGD). The struc
 ture and parameterization of a neural network can strongly affect the cond
 itioning of the optimization problem which greatly influences the performa
 nce of SGD and related methods. My research interests lie in understanding
  how neural architecture impacts the optimization dynamics and developing 
 more robust optimization methods that account for the structure of the neu
 ral network.\n\nBackground papers\nWan\, R.\, Zhu\, Z.\, Zhang\, X. and Su
 n\, J.\, 2021. Spherical Motion Dynamics: Learning Dynamics of Normalized 
 Neural Network using SGD and Weight Decay. Advances in Neural Informatio
 n Processing Systems\, 34.\nhttps://proceedings.neurips.cc/paper/2021/has
 h/326a8c055c0d04f5b06544665d8bb3ea-Abstract.html\n\nNeyshabur\, B.\, Salak
 hutdinov\, R.R. and Srebro\, N.\, 2015. Path-sgd: Path-normalized optimiza
 tion in deep neural networks. Advances in neural information processing 
 systems\, 28.\nhttps://proceedings.neurips.cc/paper/2015/hash/eaa32c96f62
 0053cf442ad32258076b9-Abstract.html\n\nDauphin\, Y.\, De Vries\, H. and Be
 ngio\, Y.\, 2015. Equilibrated adaptive learning rates for non-convex opti
 mization. Advances in neural information processing systems\, 28.\nhttps
 ://proceedings.neurips.cc/paper/2015/hash/430c3626b879b4005d41b8a46172e0c0
 -Abstract.html\n 
LOCATION:BC 010 https://plan.epfl.ch/?room==BC%20010
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR