A theoretical analysis of machine learning and partial differential equations

Computational Mathematics Seminar

Abstract :
Novel machine learning techniques based on deep learning have achieved remarkable results in many areas such as image classification and speech recognition. As a result, many scholars have started using them in areas which are not traditionally associated with machine learning. For instance, more and more researchers are employing deep neural networks to develop tools for the discretisation and solution of partial differential equations. Two reasons can be identified to be the driving forces behind the increased interest in neural networks in the area of the numerical analysis of PDEs. On the one hand, powerful approximation theoretical results have been established which demonstrate that neural networks can represent functions from the most relevant function classes with a minimal number of parameters. On the other hand, highly efficient machine learning techniques for the training of these networks are now available and can be used as a black box. In this talk, we will give an overview of some approaches towards the numerical
treatment of PDEs with neural networks and study the two aspects above. We will recall classical and some novel approximation theoretical results and tie these results to PDE discretisation. Additionally, we will present theoretical results that show that neural networks can very efficiently solve parametric PDEs without curse of dimension if these parametric PDEs admit a sufficiently small reduced basis. 
Providing a counterpoint, we analyse the structure of network spaces and deduce considerable problems for the black box solver. In particular, we will identify a number of structural properties of the set of neural networks that render optimisation over this set especially challenging and sometimes impossible.