BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:IC Colloquium: Opening the Black Box: Towards Theoretical Understa
 nding of Deep Learning
DTSTART:20210215T140000
DTEND:20210215T150000
DTSTAMP:20260509T225619Z
UID:ba1006f16c03a2d14530720f43aa750fc3c783723c7181dbce960b04
CATEGORIES:Conferences - Seminars
DESCRIPTION:By: Wei Hu - Princeton University\nIC Faculty candidate\n\nAbs
 tract\nDespite the phenomenal empirical successes of deep learning in many
  application domains\, its underlying mathematical mechanisms remain poorl
 y understood. Mysteriously\, deep neural networks in practice can often fi
 t training data perfectly and generalize remarkably well to unseen test da
 ta\, despite highly non-convex optimization landscapes and significant ove
 r-parameterization. Moreover\, deep neural networks show extraordinary abi
 lity to perform representation learning: feature representation extracted 
 from a trained neural network can be useful for other related tasks.\n\nIn
  this talk\, I will present our recent progress on building the theoretica
 l foundations of deep learning\, by opening the black box of the interact
 ions among data\, model architecture\, and training algorithm. First\, 
 I will show that gradient descent on deep linear neural networks induces a
 n implicit regularization effect towards low rank\, which explains the sur
 prising generalization behavior of deep linear networks for the low-rank m
 atrix completion problem. Next\, turning to nonlinear deep neural networks
 \, I will talk about a line of studies on wide neural networks\, where by 
 drawing a connection to the neural tangent kernels\, we can answer various
  questions such as how training loss is minimized\, why trained network ca
 n generalize\, and why certain component in the network architecture is us
 eful\; we also use theoretical insights to design a new simple and effecti
 ve method for training on noisily labeled datasets. Finally\, I will analy
 ze the statistical aspect of representation learning\, and identify key da
 ta conditions that enable efficient use of training data\, bypassing a kno
 wn hurdle in the i.i.d. tasks setting.\n\nBio\nWei Hu is a PhD candidate i
 n the Department of Computer Science at Princeton University\, advised by 
 Sanjeev Arora. Previously\, he obtained his B.E. in Computer Science from 
 Tsinghua University. He has also spent time as a research intern at resear
 ch labs of Google and Microsoft. His current research interest is broadly 
 in the theoretical foundations of modern machine learning. In particular\,
  his main focus is on obtaining solid theoretical understanding of deep le
 arning\, as well as using theoretical insights to design practical and pri
 ncipled machine learning methods. He is a recipient of the Siebel Scholars
 hip Class of 2021.\n\nMore information
LOCATION:https://epfl.zoom.us/j/83953364252?pwd=a0NWQXlDRGNMU0dZZ1BJdUdrMG
 14UT09
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR