Learning with Sparsity and Scattering Networks

Abstract

The past decades have witnessed dramatic progress in artificial intelligence. Machine

learning technology now powers many aspects of modern life, but the challenges

remain. New learning algorithms and architectures need to be developed to take advantage

of the increases in the amount of available computation and data.

This thesis focuses on two of the most important guiding principles regarding

learning algorithms, sparsity and depth. Sparsity leads to efficient and compact representations. It is a key element in domains including statistics, signal processing and

machine learning. Depth allows discovering intricate structure and learning high-level

abstraction. It is the reason for many recent breakthroughs in processing images,

video, speech, audio and texts. By designing learning and classification algorithms

that combine sparsity and depth, we show that these two concepts benefit each other

and bring about improvements of classification performance.

We first show the effectiveness of combining sparse representation-based classification with a special type of deep convolutional networks called scattering, in the

context of music genre classication. Then we propose an unsupervised deep learning

model with sparsity criteria for the classication of high-dimensional unstructured

data. In both cases, variants of strategies and architectures are presented, and state-of-the-art results are obtained.