Dualities in Neural Networks

Dualities give different descriptions of the same physical system that can allow for progress in one duality frame that is not possible in another. In this talk I'll introduce dualities in neural networks. In the first case in appropriate large-N limits, neural networks are functions drawn from Gaussian processes, allowing a perturbative non-Gaussian approach at finite N, akin to quantum field theory. In this limit the parameter space complexity formally goes to infinity, while the function space perspective analogous to effective field theory becomes more tractable. Yet these are dual descriptions of the same system, and I will exploit duality to deduce symmetries of the function-space action even when it is not explicitly known. Time permitting, I will also demonstrate analogs of so-called IR dualities in neural networks, leading to notions of universality via the study of correlation functions.