BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:CECAM workshop: "From machine-learning theory to driven complex sy
stems and back"
DTSTART;VALUE=DATE:20240522
DTSTAMP;VALUE=DATE-TIME:20240518T115042Z
UID:ac44b97f184995a8fce489f91a371d4e3f9720a31aaba721868bff82
CATEGORIES:Conferences - Seminars
DESCRIPTION:You can apply to participate and find all the relevant informa
tion (speakers\, abstracts\, program\,...) on the event website: https://w
ww.cecam.org/workshop-details/1287\n\nDescription\nIn this workshop\, we p
ropose to gather researchers with complementary backgrounds\, all involved
in cutting-edge research in statistical physics\, machine learning and st
atistical inference. The goal of this workshop is to strengthen the links
between machine learning (ML)\, disordered systems and driven complex syst
ems - such as structural glasses and dense active matter - to mutually ex
ploit their theoretical and computational tools as well as their physical
intuition. Our main focus will be on stochastic dynamical processes\, out-
of-equilibrium regimes and their insights into training dynamics\, primari
ly from a computational perspective. In addition to deepening our theoreti
cal understanding of the successes and limitations of ML\, these connectio
ns will pave the way for the development of new algorithms and suggest alt
ernative architectures. \nWe plan to address specifically the following t
opics:\n\n Dynamical Mean-Field Theory (DMFT)\n Generative neural networks
for modeling\n Phase diagrams\, landscapes and training optimization\n\nM
achine learning (ML) has become ubiquitous in the last decade. Many everyd
ay tasks can now be accomplished with ML-assisted tools\, such as ChatGPT
as a writing assistant\, Copilot as a programming assistant\, or image-gen
erating models for art and designs. Due to its strong impact on both indus
try and fundamental science\, ML has become an extremely active research a
rea\, leading to lively exchanges between practitioners and theorists in v
ery diverse communities. Its great success requires a deeper theoretical u
nderstanding and integrating complementary expertise to address its many c
hallenges [1\,2\,3].\nTraining a ML model with a particular architecture o
n a dataset amounts to evolving the parameters in a complex high-dimension
al landscape defined by a given loss function. The main questions that ari
se are: (i) how the landscape statistical properties depend on the archite
cture and the dataset statistics\, (ii) what is the associated performance
of standard optimization algorithms such as stochastic gradient descent\,
(iii) how these algorithms can be improved in terms of generalization and
computational efficiency\, (iv) what is the impact of the dataset statist
ics on the learning process. In terms of modeling\, a particular challenge
is to design correlated artificial datasets to study the learning dynamic
s in a controlled manner. Moreover\, important insights into either the le
arning process or practical applications should come from the interpretabi
lity of the learned parameters. \nWhile it is obvious that deep neural ne
tworks are capable of handling increasingly complicated tasks\, understand
ing how the formation of complex patterns relates to the dataset statistic
al properties is highly non-trivial\, even for relatively simple architect
ures. Recent advances include the construction of novel loss functions aim
ed at accelerating the learning [4]\, the development of synthetic dataset
s with a higher degree of complexity capable of mimicking real datasets [5
]\, or investigating the structure of the underlying complex landscapes [6
]. In parallel\, out-of-equilibrium physics has proven particularly useful
in developing and controlling powerful generative models that can fully d
escribe the variety of complex datasets [7\,8\,9\,10]. These studies had a
strong impact on the computational level. However\, the development of ne
w algorithms is often guided by intuitions about the loss-function specifi
c properties\, and a comprehensive understanding of the learning dynamics
is still lacking. \nRecent efforts in this direction rely on studying Lan
gevin equations associated with simple models. The formalism of dynamical
mean-field theory (DMFT)\, developed in the context of statistical physics
to study the out-of-equilibrium dynamics of structural glasses [8\,11] an
d even dense active systems [12]\, has been adapted to inference and ML mo
dels [13\,14]. Its numerical implementation poses challenges that must be
overcome to fully exploit it for improving the training process [15\,16].\
nFinally\, generative neural networks have great potential for modeling co
mplex data. One approach is energy-based modeling\, in which the probabili
ty distribution is represented by a Boltzmann distribution with a neural n
etwork as the energy function. Interpreting the trained neural network as
a disordered interaction Hamiltonian is a powerful tool for inference appl
ications [17\,18]. However\, further research is needed to understand how
the dataset patterns are encoded in the model\, and how to interpret them.
\n\nReferences\n[1] Davide Carbone\, Mengjian Hua\, Simon Coste\, Eric Van
den-Eijnden\, "Efficient Training of Energy-Based Models Using Jarzynski E
quality"\, arXiv:2305.19414 [cs.LG]\n[2] A. Muntoni\, A. Pagnani\, M. Weig
t\, F. Zamponi\, BMC Bioinformatics\, 22\, 528 (2021)\n[3] S. Cocco\, C.
Feinauer\, M. Figliuzzi\, R. Monasson\, M. Weigt\, Rep. Prog. Phys.\, 81\
, 032601 (2018)\n[4] Cedric Gerbelot\, Emanuele Troiani\, Francesca Mignac
co\, Florent Krzakala\, Lenka Zdeborova\, "Rigorous dynamical mean field t
heory for stochastic gradient descent methods"\, arXiv:2210.06591 [math-ph
]\n[5] A. Manacorda\, G. Schehr\, F. Zamponi\, The Journal of Chemical Phy
sics\, 152\, (2020)\n[6] S. Sarao Mannelli\, G. Biroli\, C. Cammarota\, F
. Krzakala\, P. Urbani\, L. Zdeborová\, Phys. Rev. X\, 10\, 011057 (2020
)\n[7] F. Mignacco\, F. Krzakala\, P. Urbani\, L. Zdeborová\, J. Stat. Me
ch.\, 2021\, 124008 (2021)\n[8] P. Morse\, S. Roy\, E. Agoritsas\, E. Sta
nifer\, E. Corwin\, M. Manning\, Proc. Natl. Acad. Sci. U.S.A.\, 118\, (2
021)\n[9] E. Agoritsas\, T. Maimbourg\, F. Zamponi\, J. Phys. A: Math. The
or.\, 52\, 144002 (2019)\n[10] G. Carleo\, I. Cirac\, K. Cranmer\, L. Dau
det\, M. Schuld\, N. Tishby\, L. Vogt-Maranto\, L. Zdeborová\, Rev. Mod.
Phys.\, 91\, 045002 (2019)\n[11] Elisabeth Agoritsas\, Giovanni Catania\,
Aurélien Decelle\, Beatriz Seoane\, "Explaining the effects of non-conve
rgent sampling in the training of Energy-Based Models"\, arXiv:2301.09428
[cs.LG] / ICML2023\n[12] E. Agoritsas\, G. Biroli\, P. Urbani\, F. Zamponi
\, J. Phys. A: Math. Theor.\, 51\, 085002 (2018)\n[13] Jascha Sohl-Dickst
ein\, Eric A. Weiss\, Niru Maheswaranathan\, Surya Ganguli\, "Deep Unsuper
vised Learning using Nonequilibrium Thermodynamics"\, arXiv:1503.03585 [cs
.LG]\n[14] Carlo Lucibello\, Marc Mézard\, "The Exponential Capacity of D
ense Associative Memories"\, arXiv:2304.14964 [cond-mat.dis-nn]\n[15] S. G
oldt\, M. Mézard\, F. Krzakala\, L. Zdeborová\, Phys. Rev. X\, 10\, 041
044 (2020)\n[16] Miguel Ruiz-Garcia\, Ge Zhang\, Samuel S. Schoenholz\, An
drea J. Liu\, "Tilting the playing field: Dynamical loss functions for mac
hine learning"\, ICML 2021 / arXiv:2102.03793 [cs.LG]\n[17] A. Decelle\, P
hysica A: Statistical Mechanics and its Applications\, 128154 (2022)\n[18]
Chapter 24 to P. Charbonneau et al.\, "Spin Glass Theory and Far Beyond:
Replica Symmetry Breaking after 40 Years"\, World Scientific (2023)
LOCATION:BCH 2103 https://plan.epfl.ch/?room==BCH%202103
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR