ORIGINAL

Abstract

Inspired by the potential synergy between models of visual categorization from cognitive science and neural classifiers from computer vision, this project proposes a probabilistic framework that integrates the mechanism of cognitively-inspired generative classification into an otherwise discriminative deep learning system. Specifically, we integrate a Gaussian classifier into a Convolutional Neural Network, jointly learning embeddings of images and distributions over embeddings. We dub these models Deep Prototype Models (DPMs), and find that they boost validation accuracy and reduce generalization loss under modest distributional shift. Additionally, we examine their similarity to human categorization behavior across two dimensions – the uncertainty or relative activation across categories for stimuli, and the organization of stimuli within categories. For the former, we employ an existing dataset of full-label distributions of human categorizations, and find that DPMs provide a better fit to human uncertainty behavior. In the latter case, we explore typicality as a method for inferring the structure of categories from human behavior, and collect a novel large-scale dataset of over 350,000 typicality judgements. We find a number of interesting relationships between typicality, response time, and human agreement; and find that standard CNNs correlate better to human typicality judgements than DPMs. The DPMs proposed and typicality dataset collected represent an initial effort to bridge the gap between cognitive modelling and modern deep learning.