BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:Advances in Mathematics of Deep Learning
DTSTART:20230707T130000
DTEND:20230707T150000
DTSTAMP:20260603T174114Z
UID:000f5ecc08566a41b9dc3ed300f0c749ebf6403297134fb38c3c3409
CATEGORIES:Conferences - Seminars
DESCRIPTION:Hristo Papazov\nEDIC candidacy exam\nExam president: Prof. Nic
 olas Boumal\nThesis advisor: Prof. Nicolas Flammarion\nCo-examiner: Prof. 
 Lénaïc Chizat\n\nAbstract\nIn this research proposal\, we will be consid
 ering the question of how -- for a fixed objective function\, model class\
 , and optimization algorithm -- different parametrizations of the model an
 d different initializations of the parameters influence the optimization t
 rajectory and the generalization properties of the learning procedure. In 
 other words\, we will be investigating the implicit bias induced by the ch
 oice of parametrization and initialization\, and the following three paper
 s will guide our discussion:\n    -- "An Asymptotical Variational Princ
 iple Associated with the Steepest Descent Method for a Convex Function" by
  Lemaire\;\n    -- "Kernel and Rich Regimes in Overparametrized Models"
  by Woodworth et al.\;\n    -- "Implicit Bias of Gradient Descent on Re
 parametrized Models: On Equivalence to Mirror Descent" by Li et al.\nAs a 
 rough roadmap for the report\, we will first develop some foundational too
 ls for analyzing the trajectory and limit of convex gradient flow through 
 Lemaire. Then\, we will study a certain class of reparametrized linear mod
 els from Woodworth et al. where the geometry and scale of the initialized 
 parameters lead to distinct optimization and generalization behavior. In t
 his setting\, we cannot directly apply the techniques from Lemaire because
  we are optimizing over a nonconvex loss. However\, fortunately\, the opti
 mization procedure can be rephrased as a convex mirror flow\, for which al
 l of the tricks from Lemaire carry over. Finally\, we consider Li et al. w
 ho give precise necessary and sufficient conditions for when a gradient fl
 ow on a reparametrized model can be reformulated as a mirror flow with a L
 egendre function on the "effective" parameters so that we can reap the ben
 efits of a convex loss.\n\nBackground papers\n\n	An Asymptotical Variation
 al Principle Associated with the Steepest Descent Method for a Convex F
 unction: https://www.heldermann-verlag.de/jca/jca03/jca03005.pdf\n	Kernel
  and Rich Regimes in Overparametrized Models: https://arxiv.org/abs/2002.
 09277\n	Implicit Bias of Gradient Descent on Reparametrized Models: On Equ
 ivalence to Mirror Descent: https://arxiv.org/abs/2207.04036\n
LOCATION:BC 233 https://plan.epfl.ch/?room==BC%20233
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR