BMI Seminar // Friedemann Zenke - Learning World Models via Internal Prediction: From Local Plasticity to Recurrent Circuits

To guide behavior, the brain must extract object identities and their dynamics from entangled sensory streams, and use this information to anticipate what will happen next. How it builds such internal models from experience remains an open question. I will briefly recap earlier work showing that a local, predictive form of Hebbian plasticity allows neural networks to learn invariant object representations without supervision, using learning rules consistent with known properties of synaptic plasticity.

The main part of the talk will focus on how predictive principles scale from invariance learning to circuit-level computation to yield internal world models. I will first introduce a theoretical framework, latent distribution matching (LDM), that casts self-supervised learning as balancing predictive alignment with entropy-driven uniformity in latent space. LDM unifies a broad class of existing self-supervised objectives, including contrastive, non-contrastive, and joint-embedding predictive approaches, and yields identifiability guarantees for nonlinear predictive models. Building on these principles, I will present recurrent predictive learning (RPL), a self-supervised joint-embedding predictive architecture that learns untangled representations of objects and their dynamics directly from sensory streams, without reconstruction or labels, and that develops successor-like and abstract sequence representations reminiscent of those found in visual and prefrontal cortex. Finally, I will show how these ideas extend to model-based reinforcement learning through a reconstruction-free variant of Dreamer, in which a JEPA-style predictor operating on continuous, deterministic representations matches the performance of reconstruction-based world models.

Together, these results point toward an emerging framework in which internal prediction is the driver of plasticity. However, for the brain to tap into such powerful representation learning, local learning seems too limited and we have to rethink plasticity mechanisms that act at the circuit level.