IC Colloquium: Automatic differentiation for dummies
Event details
| Date | 21.01.2019 |
| Hour | 16:15 › 17:30 |
| Location | |
| Category | Conferences - Seminars |
By: Simon Peyton Jones - Microsoft Research (Cambridge)
Video of his talk
Abstract:
Automatic differentiation (AD) is clearly cool. And it has become useful (as well as cool) recently, because it essentially implements the back-propagation step that is key to learning in neural networks; and it is used extensively in many other machine learning contexts. But as well as been cool and useful, AD is also mysterious. There are forward and backward variants (the latter being mind-bending); there are tapes and mutation, delimited continuations, and more. It all makes me feel stupid.
In this talk, I’ll explain a simple and elegant approach to AD from first principles, based on work by Conal Elliot. We’ll end up with an AD algorithm that takes 10 lines of code; that works for forward and backward AD; that works for matrix computations; and that is a source-to-source transformation whose output can be efficiently compiled without needing a runtime trace.
Bio:
Simon Peyton Jones, FRS, graduated from Trinity College Cambridge in 1980. After two years in industry, he spent seven years as a lecturer at University College London, and nine years as a professor at Glasgow University, before moving to Microsoft Research (Cambridge) in 1998. Simon’s main research interest is in functional programming languages, their implementation, and their application. He was a key contributor to the design of the now-standard functional language Haskell, and is the lead designer of the widely-used Glasgow Haskell Compiler (GHC). He has written two textbooks about the implementation of functional languages. He is particularly motivated by direct application of principled theory to practical language design and implementation — that is one reason he loves functional programming so much. Simon is chair of Computing at School, the grass-roots organisation that was at the epicentre of the 2014 reform of the English computing curriculum.
More information
Video of his talk
Abstract:
Automatic differentiation (AD) is clearly cool. And it has become useful (as well as cool) recently, because it essentially implements the back-propagation step that is key to learning in neural networks; and it is used extensively in many other machine learning contexts. But as well as been cool and useful, AD is also mysterious. There are forward and backward variants (the latter being mind-bending); there are tapes and mutation, delimited continuations, and more. It all makes me feel stupid.
In this talk, I’ll explain a simple and elegant approach to AD from first principles, based on work by Conal Elliot. We’ll end up with an AD algorithm that takes 10 lines of code; that works for forward and backward AD; that works for matrix computations; and that is a source-to-source transformation whose output can be efficiently compiled without needing a runtime trace.
Bio:
Simon Peyton Jones, FRS, graduated from Trinity College Cambridge in 1980. After two years in industry, he spent seven years as a lecturer at University College London, and nine years as a professor at Glasgow University, before moving to Microsoft Research (Cambridge) in 1998. Simon’s main research interest is in functional programming languages, their implementation, and their application. He was a key contributor to the design of the now-standard functional language Haskell, and is the lead designer of the widely-used Glasgow Haskell Compiler (GHC). He has written two textbooks about the implementation of functional languages. He is particularly motivated by direct application of principled theory to practical language design and implementation — that is one reason he loves functional programming so much. Simon is chair of Computing at School, the grass-roots organisation that was at the epicentre of the 2014 reform of the English computing curriculum.
More information
Practical information
- General public
- Free
- This event is internal
Contact
- Host: Christoph Koch