Generating invariant representations with cortex-inspired models of unsupervised learning

Abstract

It is generally believed that the learning capabilities of real biological brains are unmatched by modern artificial intelligence systems. However state of the art artificial systems usually do not closely resemble more biologically inspired models of learning. For visual object recognition, typical state of the art systems are convolutional networks trained by backpropagating errors from millions of labeled images. Many biological models do not rely on backpropagation of supervised error signals, and instead use unsupervised local learning rules to update synaptic connections. In this thesis we investigate the recognition capabilities of one such biological model, sparse coding. We then propose novel methods to improve recognition capabilities of bio-inspired models using unsupervised local learning rules. In chapter 1 we discuss previous work on biologically-inspired models of visual cortex. In chapter 2 we discuss quantitative methods to evaluate such models by measuring the invariance properties of their generated neural responses. In chapter 3 we examine the popular learning model, sparse coding, from the perspective of invariant object recognition. We argue that the generated neural responses are actually less invariant than pixels to small input distortions and speculate that this is undesirable for invariant object recognition. In chapter 4 we propose another algorithm, kernel similarity matching, based on local learning rules that learns sparse high dimensional representations which preserves the relative ordering of similarities between pairs of patterns. Finally in chapter 5 we propose another algorithm, invariant subspace modules, based off applying subspace clustering to image patches. Computing the norm of an input patch inside these learned subspaces can be regarded as invariant feature detection. Coupled with proper normalization, subspace clustering can be applied in a hierarchical fashion, leading to representations which support classification, at least for simple handwritten MNIST digits.