IC Colloquium: Information Lattice Learning
By: Lav Varshney - Stony Brook University
Video of his talk
Abstract
Drawing on group-theoretic and information-theoretic foundations that go back to Shannon's lattice theory of information, we propose information lattice learning (ILL) as a general framework to learn rules of a signal. In our definition, a rule is a coarsened signal used to help us gain interpretable insights about the original signal. To make full sense of what might govern the signal’s intrinsic structure, we seek multiple disentangled rules arranged in a hierarchy, called a lattice. Compared to representation/rule-learning models optimized for a specific task (e.g., classification), ILL focuses on explainability: it is designed to mimic human experiential learning and discover rules akin to those humans can distill and comprehend. We detail the mathematical foundations and algorithms of ILL, and illustrate how it addresses the fundamental question “what makes X an X” by creating rule-based explanations designed to help humans understand. We show ILL’s efficacy and interpretability on benchmarks and assessments in visual classification, significantly outperforming vision transformers. We further show optimality in semantic compression and the ability to create formally-verifiable digital twins. We close with applications in knowledge discovery and creativity, using ILL to distill music theory from scores and enabling a new way to compose music, as well as some early work on understanding the principles that govern scattering amplitudes in Super Yang-Mills theory, rather than just predicting them.
Bio
Lav R. Varshney is the Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute at Stony Brook University. He is co-founder and CEO of Kocree, Inc., a startup company building novel human-controllable AI for discovery and creativity, and chief scientist of Ensaras, Inc., a startup company focused on AI and wastewater treatment. He holds appointments at RAND Corporation and at Brookhaven National Laboratory. He was previously on the faculty of the University of Illinois Urbana-Champaign, a visiting scholar at Northwestern's Kellogg School of Management, a principal research scientist at Salesforce Research AI where he was part of the team that developed and open weight released the first billion-parameter large language model, and a research staff member at IBM Research where he led the design and deployment of the first commercially-successful generative AI technology. He is a former White House staffer, having served on the National Security Council staff as a White House Fellow, where he contributed to national/international AI and wireless communications policy. His research interests include information theory and artificial intelligence. He received his B.S. degree from Cornell University and his S.M. and Ph.D. degrees from the Massachusetts Institute of Technology. He studied at EPFL in 2006 with Emre Telatar and Ruediger Urbanke.
More information
Video of his talk
Abstract
Drawing on group-theoretic and information-theoretic foundations that go back to Shannon's lattice theory of information, we propose information lattice learning (ILL) as a general framework to learn rules of a signal. In our definition, a rule is a coarsened signal used to help us gain interpretable insights about the original signal. To make full sense of what might govern the signal’s intrinsic structure, we seek multiple disentangled rules arranged in a hierarchy, called a lattice. Compared to representation/rule-learning models optimized for a specific task (e.g., classification), ILL focuses on explainability: it is designed to mimic human experiential learning and discover rules akin to those humans can distill and comprehend. We detail the mathematical foundations and algorithms of ILL, and illustrate how it addresses the fundamental question “what makes X an X” by creating rule-based explanations designed to help humans understand. We show ILL’s efficacy and interpretability on benchmarks and assessments in visual classification, significantly outperforming vision transformers. We further show optimality in semantic compression and the ability to create formally-verifiable digital twins. We close with applications in knowledge discovery and creativity, using ILL to distill music theory from scores and enabling a new way to compose music, as well as some early work on understanding the principles that govern scattering amplitudes in Super Yang-Mills theory, rather than just predicting them.
Bio
Lav R. Varshney is the Della Pietra Infinity Professor and inaugural director of the AI Innovation Institute at Stony Brook University. He is co-founder and CEO of Kocree, Inc., a startup company building novel human-controllable AI for discovery and creativity, and chief scientist of Ensaras, Inc., a startup company focused on AI and wastewater treatment. He holds appointments at RAND Corporation and at Brookhaven National Laboratory. He was previously on the faculty of the University of Illinois Urbana-Champaign, a visiting scholar at Northwestern's Kellogg School of Management, a principal research scientist at Salesforce Research AI where he was part of the team that developed and open weight released the first billion-parameter large language model, and a research staff member at IBM Research where he led the design and deployment of the first commercially-successful generative AI technology. He is a former White House staffer, having served on the National Security Council staff as a White House Fellow, where he contributed to national/international AI and wireless communications policy. His research interests include information theory and artificial intelligence. He received his B.S. degree from Cornell University and his S.M. and Ph.D. degrees from the Massachusetts Institute of Technology. He studied at EPFL in 2006 with Emre Telatar and Ruediger Urbanke.
More information
Practical information
- General public
- Free
Contact
- Host: Haitham Al Hassanieh