An exposition of memory-augmented artificial neural networks

Biologically inspired machine learning algorithms known as artificial neural networks have been around since the 1950s, but have rapidly gained popularity over the last decade due to increases in computational power and the sheer volume of data available. The simplest forms of these, feed-forward neural networks, have two major shortcomings: they require input of fixed length and handle each input independently. As such these algorithms have no "memory" and are unsuitable for sequence learning problems. Simple variants, recurrent neural networks, equip the neural network with a basic internal memory and have shown success in handling such problems. However, this memory is inherently short-lived and these networks struggle to learn longer term dependencies. A very recent biologically inspired variant, neural Turing machines, augment the network with an external memory and show potential to address such problems.