FashionPolice: Application of Convolutional Neural Networks to Single-Step Apparel Recognition on Social Media in Scarce Training Data Contexts

Abstract

Over the past fifteen year, the field of object recognition has evolved from the limited invariant alignment methods of the 1960's and 1970's, to neural network based vision techniques nowadays. Today's state of the art algorithms perform simple recognition tasks, such as the MNIST handwritten digit classification challenge, with virtually perfect success rates–they are better than humans. Some other areas in computer vision lag behind, however.This thesis addresses the challenge of recognizing fashion apparel as it appears in social media images. Its goal is to build software that automatically detects and keeps track of a brand's products in images posted on social media through a single-step recognition process, implemented by a convolutional neural network able to adapt quickly, and inexpensively, to new collections. Social media in general, and bloggers in particular, are crucial drivers of consumption in fashion: in 2016, nearly one in two online shoppers declared being influenced by posts on social media before making purchases, while only one in three declared being influenced by ads. This statistic is made all the more relevant by the $51.5 billion spent on online purchases of apparel and accessories in the U.S. in 2015 (the biggest share of all online spending that year).This paper shifts away from the widely adopted two-step "detection and classification" method, and bypasses the often used descriptive label-assigning classification paradigm and its associated nearest-neighbor recommendation. Instead, the focus is turned to recognizing each item in a brand's collection as an instance of a single class. Implementing techniques in transfer learning, residual learning, optimal learning acceleration, and training data transformation, the FashionPolice model developed in this thesis achieves above-baseline performance while maintaining computational load at a minimum.This is a business-oriented project; commercial applications could enable improvements to online demand modeling, online supply chain control, and provide additional marketing insights.