Emergent Capabilities in Modern Sequence Models: Phase Transitions, Memory in Shallow Transformers, and Bidirectional State-Space Architectures.
Event details
| Date | 17.06.2025 |
| Hour | 14:00 › 16:00 |
| Speaker | Fabrizio Boncoraglio |
| Location | |
| Category | Conferences - Seminars |
EDIC candidacy exam
Exam president: Prof. Michael Gastpar
Thesis advisor: Prof. Lenka Zdeborova
Co-examiner: Prof. Matthieu Wyart
Abstract
coming soon
Selected papers
- A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention: https://arxiv.org/pdf/2402.03902
- Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers: https://arxiv.org/pdf/2407.09941
- Understanding Factual Recall in Transformers via Associative Memories: https://arxiv.org/pdf/2412.06538
Exam president: Prof. Michael Gastpar
Thesis advisor: Prof. Lenka Zdeborova
Co-examiner: Prof. Matthieu Wyart
Abstract
coming soon
Selected papers
- A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention: https://arxiv.org/pdf/2402.03902
- Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers: https://arxiv.org/pdf/2407.09941
- Understanding Factual Recall in Transformers via Associative Memories: https://arxiv.org/pdf/2412.06538
Practical information
- General public
- Free
Contact
- edic@epfl.ch