BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Memento EPFL//
BEGIN:VEVENT
SUMMARY:AI Center Seminar - AI Fundamentals series - Anej Svete - "Express
 ivity-Maxxing: What To Do When Your Transformer Is Giving Mid"
DTSTART:20261022T110000
DTEND:20261022T120000
DTSTAMP:20261022T041747Z
UID:36fd40cc114176bf34cf46e7c1a3967bef9be3d4010f8e75471e2299
CATEGORIES:Conferences - Seminars
DESCRIPTION:Anej Svete\nThe talk is jointly organized by the EPFL AI Cen
 ter and the TML Lab as part of the AI fundamentals seminar series.\n\nHo
 sting Professor: Prof. Nicolas Flammarion\n\nTitle\nExpressivity-Maxxing: 
 What To Do When Your Transformer Is Giving Mid\nImproving the Reasoning Ab
 ilities of Transformers with Looping\, Latent Reasoning\, and Diffusion\n\
 nAbstract\nTransformers famously compute with a fixed number of parallel s
 teps\, one per layer. That makes them fast\, but it also means that there 
 are very simple problems transformers cannot solve. The cup shuffling game
  where one has to follow a ball under a cup while the cups are shuffled is
  a classic example. Chain-of-thought reasoning makes transformers much mor
 e powerful\, but it is slow: The model thinks one token at a time and has 
 to squeeze intermediate result into words. This talk is a tour of the ways
  to give a transformer more computation without making it talk: continuous
  thoughts\, looping\, and diffusion. Continuous thoughts let a transformer
  run textbook algorithms efficiently\, looping reaches progressively harde
 r classes of parallelizable problems\, and masked diffusion models turn ou
 t to be padded looped transformers in disguise. I will close with a recent
 ly proposed way of training a looped transformer to reason in parallel tha
 t comes close to explicit chain of thought while thinking 2.5 times faster
 .\n\nBio\nAnej Svete is a doctoral fellow at the ETH AI Center\, co-advise
 d by Ryan Cotterell and Valentina Boeva. He studies what language models c
 an and cannot compute\, using formal language and complexity theory. He us
 es those results for theory-driven architecture design: Making transformer
 s more powerful with looping or by designing hybrid models that combine th
 e strengths of transformers and linear recurrent neural networks. He holds
  a bachelor's degree in computer science and mathematics from the Universi
 ty of Ljubljana and a master's degree in data science from ETH Zürich.
LOCATION:ELE 117 https://plan.epfl.ch/?room==ELE%20117 https://epfl.zoom.u
 s/j/64785692648?pwd=UqaqXRbEauTTBv9RGAep9IR0xAJpzw.1
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR