IC Colloquium: Advancing System Models of Brain-Like Intelligence via Integrative Benchmarking
By: Martin Schrimpf - MIT
IC/SV Faculty candidate
Abstract
Efforts in machine learning attempt to engineer artificially intelligent models, while research in the brain and cognitive sciences attempts to uncover the neural circuitry underlying human intelligence in domains such as vision and language. I will argue that the time is ripe (again) for rich synergies between both fields: build system models that capture neural mechanisms and supported behaviors within entire domains of intelligence. To make progress on system models, I propose integrative benchmarking – integrating experimental results from many laboratories into suites of benchmarks that guide and constrain those models at multiple stages and scales. We show-case this approach by developing Brain-Score benchmark suites for neural and behavioral experiments in the primate visual ventral stream as well as the human language system. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (~50% explained variance), but also discover key relationships: Models’ brain scores are predicted by their object categorization performance in vision (but only up to 70% ImageNet accuracy), and their next-word prediction performance in language. The better models predict internal neural activity, the better they match human behavioral outputs, with architecture substantially contributing to brain-like representations. Using the integrative benchmarks, we develop improved state-of-the-art system models that more closely match shallow recurrent neuroanatomy, predict primate temporal processing, and are more robust to image corruptions. Taken together, these integrative benchmarks and system models are first steps to modeling the complexities of brain processing in entire domains of intelligence.
Bio
Martin is a research scientist at the MIT Quest for Intelligence, where he bridges research in natural and artificial intelligence. He focuses on computational models of vision and language that are neurally and behaviorally aligned with primates. Martin got his PhD from MIT, working with James DiCarlo on the neural mechanisms underlying object recognition. Before, he worked on natural language processing with Richard Socher at Salesforce Research, and on human pattern completion with Gabriel Kreiman at Harvard. Martin got his Bachelor and Master degrees from TUM, LMU, and UNA in Germany. He co-founded the non-profit startup Integreat, and previously worked at Siemens and Oracle Research.
More information
IC/SV Faculty candidate
Abstract
Efforts in machine learning attempt to engineer artificially intelligent models, while research in the brain and cognitive sciences attempts to uncover the neural circuitry underlying human intelligence in domains such as vision and language. I will argue that the time is ripe (again) for rich synergies between both fields: build system models that capture neural mechanisms and supported behaviors within entire domains of intelligence. To make progress on system models, I propose integrative benchmarking – integrating experimental results from many laboratories into suites of benchmarks that guide and constrain those models at multiple stages and scales. We show-case this approach by developing Brain-Score benchmark suites for neural and behavioral experiments in the primate visual ventral stream as well as the human language system. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (~50% explained variance), but also discover key relationships: Models’ brain scores are predicted by their object categorization performance in vision (but only up to 70% ImageNet accuracy), and their next-word prediction performance in language. The better models predict internal neural activity, the better they match human behavioral outputs, with architecture substantially contributing to brain-like representations. Using the integrative benchmarks, we develop improved state-of-the-art system models that more closely match shallow recurrent neuroanatomy, predict primate temporal processing, and are more robust to image corruptions. Taken together, these integrative benchmarks and system models are first steps to modeling the complexities of brain processing in entire domains of intelligence.
Bio
Martin is a research scientist at the MIT Quest for Intelligence, where he bridges research in natural and artificial intelligence. He focuses on computational models of vision and language that are neurally and behaviorally aligned with primates. Martin got his PhD from MIT, working with James DiCarlo on the neural mechanisms underlying object recognition. Before, he worked on natural language processing with Richard Socher at Salesforce Research, and on human pattern completion with Gabriel Kreiman at Harvard. Martin got his Bachelor and Master degrees from TUM, LMU, and UNA in Germany. He co-founded the non-profit startup Integreat, and previously worked at Siemens and Oracle Research.
More information
Practical information
- General public
- Free
Contact
- Host: Alexander Mathis