BEGIN:VCALENDAR VERSION:2.0 PRODID:-//Memento EPFL// BEGIN:VEVENT SUMMARY:CIS Network Seminars: #IDIAP - Dr. James Henderson DTSTART:20230424T151500 DTEND:20230424T161500 DTSTAMP:20260413T235550Z UID:966bd98e71e0f041502d05b7f3d09b4f4f315668d9e0adacf8dbec6d CATEGORIES:Conferences - Seminars DESCRIPTION:James Henderson\nTitle: Understanding Language as Bayesian In ference of Bayesian Representations\n\nAbstract:\nThe astounding recent ad vances in AI have been enabled by the Transformer deep learning architectu re\, and its incredibly effective inductive bias for Large Language Models . We argue that this architecture is fundamentally Bayesian\, in that its learned representations are best thought of as probability distributions. We then propose an extension to Transformers which adds nonparametric Baye sian inference of these Bayesian representations. Our Nonparametric Variat ional Information Bottleneck can regularise any attention-based representa tion\, resulting in generative models with variable-sized sparse latent re presentations. Existing large Transformers pretrained on text can be reint erpreted as approximating this inference\, suggesting that the success of Transformers is due to their ability to approximate nonparametric Bayesian inference of Bayesian representations.\n\nBio:\nDr James Henderson is a S enior Researcher at Idiap Research Institute\, where he heads the Natural Language Understanding group. He was previously MER at Univ Geneva and hel d a number of other research positions since his PhD at Univ Pennsylvania and BSc at MIT. He investigates machine learning methods for natural langu age processing\, with a long history of work on neural network structured prediction and variational Bayesian approaches to deep learning.\n\nThe Ce nter for Intelligent Systems at EPFL (CIS) is a collaboration among IC\, E NAC\, SB\; SV and STI that brings together researchers working on differen t aspects of Intelligent Systems.\n\nCollaboration and networking are cruc ial in the advancement of research and are key pillars of the CIS’ activ ities. We are thrilled to announce a new series of talks: the “CIS Netwo rk Seminars”\, where we will invite experts from our network to share th eir work.\n\n \nThe CIS seminar will take place In hybrid mode: Room INF 328 and by Zoom https://epfl.zoom.us/meeting/register/u5wkde2rqTItE9cNYqIr PJvoc7ONUV6JFesD \n\nMonday 24 April\, 2023 from 3:15 to 4:15 pm\n\nNB: Vi deo recordings of the seminars will be made available on our website and p ublished on our social media pages LOCATION:INF 328 https://plan.epfl.ch/?room==INF%20328 https://epfl.zoom.u s/meeting/register/u5wkde2rqTItE9cNYqIrPJvoc7ONUV6JFesD STATUS:CONFIRMED END:VEVENT END:VCALENDAR