An Exploration of Multiclass Multidomain Image Classification

Abstract

Convolutional neural networks (CNNs) are state of the art in image multiclass classification tasks. Often, these tasks involves more complex scenarios involving multiple attributes we want to either classify or reduce bias against. We call non-primary attributes in these tasks "domains." A problem arises where performance and fairness of our classifiers deteriorate as a result of these additional requirements, often in an unexpectedly large way.

First, we examine the intersection of this problem and the area of multi-task learning. We construct two multi-task image classification tasks: (1) predicting coarse and fine-grained labels on the CIFAR-100 dataset, and (2) predicting dataset domain origin and digit classification from the combination of the SVHN and MNIST datasets. We demonstrate the failure of an automatic method to balance performance across the classification tasks and consider explanations for this behavior.

Next, we construct a novel dataset from CIFAR-10 that provides a framework for studying, benchmarking, and mitigating bias in a multidomain setting. While the classification task appears deceptively simple, we show experimental results of novel CNN training and inference procedures that demonstrate some success toward the challenge of bias mitigation. Finally, we apply the mitigation strategies we have developed on an activity classification task on the imSitu dataset, and reveal real-world improvements on gender bias.