BEGIN:VCALENDAR VERSION:2.0 PRODID:-//Memento EPFL// BEGIN:VEVENT SUMMARY:CECAM workshop: "From machine-learning theory to driven complex sy stems and back" DTSTART;VALUE=DATE:20240522 DTSTAMP:20260407T163947Z UID:ac44b97f184995a8fce489f91a371d4e3f9720a31aaba721868bff82 CATEGORIES:Conferences - Seminars DESCRIPTION:You can apply to participate and find all the relevant informa tion (speakers\, abstracts\, program\,...) on the event website: https://w ww.cecam.org/workshop-details/1287\n\nDescription\nIn this workshop\, we p ropose to gather researchers with complementary backgrounds\, all involved in cutting-edge research in statistical physics\, machine learning and st atistical inference. The goal of this workshop is to strengthen the links between machine learning (ML)\, disordered systems and driven complex syst ems - such as structural glasses and dense active matter - to mutually ex ploit their theoretical and computational tools as well as their physical intuition. Our main focus will be on stochastic dynamical processes\, out- of-equilibrium regimes and their insights into training dynamics\, primari ly from a computational perspective. In addition to deepening our theoreti cal understanding of the successes and limitations of ML\, these connectio ns will pave the way for the development of new algorithms and suggest alt ernative architectures. \nWe plan to address specifically the following t opics:\n\n Dynamical Mean-Field Theory (DMFT)\n Generative neural networks for modeling\n Phase diagrams\, landscapes and training optimization\n\nM achine learning (ML) has become ubiquitous in the last decade. Many everyd ay tasks can now be accomplished with ML-assisted tools\, such as ChatGPT as a writing assistant\, Copilot as a programming assistant\, or image-gen erating models for art and designs. Due to its strong impact on both indus try and fundamental science\, ML has become an extremely active research a rea\, leading to lively exchanges between practitioners and theorists in v ery diverse communities. Its great success requires a deeper theoretical u nderstanding and integrating complementary expertise to address its many c hallenges [1\,2\,3].\nTraining a ML model with a particular architecture o n a dataset amounts to evolving the parameters in a complex high-dimension al landscape defined by a given loss function. The main questions that ari se are: (i) how the landscape statistical properties depend on the archite cture and the dataset statistics\, (ii) what is the associated performance of standard optimization algorithms such as stochastic gradient descent\, (iii) how these algorithms can be improved in terms of generalization and computational efficiency\, (iv) what is the impact of the dataset statist ics on the learning process. In terms of modeling\, a particular challenge is to design correlated artificial datasets to study the learning dynamic s in a controlled manner. Moreover\, important insights into either the le arning process or practical applications should come from the interpretabi lity of the learned parameters. \nWhile it is obvious that deep neural ne tworks are capable of handling increasingly complicated tasks\, understand ing how the formation of complex patterns relates to the dataset statistic al properties is highly non-trivial\, even for relatively simple architect ures. Recent advances include the construction of novel loss functions aim ed at accelerating the learning [4]\, the development of synthetic dataset s with a higher degree of complexity capable of mimicking real datasets [5 ]\, or investigating the structure of the underlying complex landscapes [6 ]. In parallel\, out-of-equilibrium physics has proven particularly useful in developing and controlling powerful generative models that can fully d escribe the variety of complex datasets [7\,8\,9\,10]. These studies had a strong impact on the computational level. However\, the development of ne w algorithms is often guided by intuitions about the loss-function specifi c properties\, and a comprehensive understanding of the learning dynamics is still lacking. \nRecent efforts in this direction rely on studying Lan gevin equations associated with simple models. The formalism of dynamical mean-field theory (DMFT)\, developed in the context of statistical physics to study the out-of-equilibrium dynamics of structural glasses [8\,11] an d even dense active systems [12]\, has been adapted to inference and ML mo dels [13\,14]. Its numerical implementation poses challenges that must be overcome to fully exploit it for improving the training process [15\,16].\ nFinally\, generative neural networks have great potential for modeling co mplex data. One approach is energy-based modeling\, in which the probabili ty distribution is represented by a Boltzmann distribution with a neural n etwork as the energy function. Interpreting the trained neural network as a disordered interaction Hamiltonian is a powerful tool for inference appl ications [17\,18]. However\, further research is needed to understand how the dataset patterns are encoded in the model\, and how to interpret them. \n\nReferences\n[1] Davide Carbone\, Mengjian Hua\, Simon Coste\, Eric Van den-Eijnden\, "Efficient Training of Energy-Based Models Using Jarzynski E quality"\, arXiv:2305.19414 [cs.LG]\n[2] A. Muntoni\, A. Pagnani\, M. Weig t\, F. Zamponi\, BMC Bioinformatics\, 22\, 528 (2021)\n[3] S. Cocco\, C. Feinauer\, M. Figliuzzi\, R. Monasson\, M. Weigt\, Rep. Prog. Phys.\, 81\ , 032601 (2018)\n[4] Cedric Gerbelot\, Emanuele Troiani\, Francesca Mignac co\, Florent Krzakala\, Lenka Zdeborova\, "Rigorous dynamical mean field t heory for stochastic gradient descent methods"\, arXiv:2210.06591 [math-ph ]\n[5] A. Manacorda\, G. Schehr\, F. Zamponi\, The Journal of Chemical Phy sics\, 152\, (2020)\n[6] S. Sarao Mannelli\, G. Biroli\, C. Cammarota\, F . Krzakala\, P. Urbani\, L. Zdeborová\, Phys. Rev. X\, 10\, 011057 (2020 )\n[7] F. Mignacco\, F. Krzakala\, P. Urbani\, L. Zdeborová\, J. Stat. Me ch.\, 2021\, 124008 (2021)\n[8] P. Morse\, S. Roy\, E. Agoritsas\, E. Sta nifer\, E. Corwin\, M. Manning\, Proc. Natl. Acad. Sci. U.S.A.\, 118\, (2 021)\n[9] E. Agoritsas\, T. Maimbourg\, F. Zamponi\, J. Phys. A: Math. The or.\, 52\, 144002 (2019)\n[10] G. Carleo\, I. Cirac\, K. Cranmer\, L. Dau det\, M. Schuld\, N. Tishby\, L. Vogt-Maranto\, L. Zdeborová\, Rev. Mod. Phys.\, 91\, 045002 (2019)\n[11] Elisabeth Agoritsas\, Giovanni Catania\, Aurélien Decelle\, Beatriz Seoane\, "Explaining the effects of non-conve rgent sampling in the training of Energy-Based Models"\, arXiv:2301.09428 [cs.LG] / ICML2023\n[12] E. Agoritsas\, G. Biroli\, P. Urbani\, F. Zamponi \, J. Phys. A: Math. Theor.\, 51\, 085002 (2018)\n[13] Jascha Sohl-Dickst ein\, Eric A. Weiss\, Niru Maheswaranathan\, Surya Ganguli\, "Deep Unsuper vised Learning using Nonequilibrium Thermodynamics"\, arXiv:1503.03585 [cs .LG]\n[14] Carlo Lucibello\, Marc Mézard\, "The Exponential Capacity of D ense Associative Memories"\, arXiv:2304.14964 [cond-mat.dis-nn]\n[15] S. G oldt\, M. Mézard\, F. Krzakala\, L. Zdeborová\, Phys. Rev. X\, 10\, 041 044 (2020)\n[16] Miguel Ruiz-Garcia\, Ge Zhang\, Samuel S. Schoenholz\, An drea J. Liu\, "Tilting the playing field: Dynamical loss functions for mac hine learning"\, ICML 2021 / arXiv:2102.03793 [cs.LG]\n[17] A. Decelle\, P hysica A: Statistical Mechanics and its Applications\, 128154 (2022)\n[18] Chapter 24 to P. Charbonneau et al.\, "Spin Glass Theory and Far Beyond: Replica Symmetry Breaking after 40 Years"\, World Scientific (2023) LOCATION:BCH 2103 https://plan.epfl.ch/?room==BCH%202103 STATUS:CONFIRMED END:VEVENT END:VCALENDAR