Extraction of Behavioral Dynamics and Learning Rules in Decision-Making Experiments

Abstract

Understanding how animals update their decision-making behavior over time is an important problem in neuroscience. Perceptual decision-making strategies evolve over the course of learning, and continue to vary even in well-trained animals. While reinforcement learning aims to understand such strategies from a normative perspective, there are comparatively fewer purely descriptive tools for capturing behavioral dynamics. Behavior is either treated as fixed or is tracked only with coarse performance statistics, providing limited insight into the evolution of decision-making strategies.

In this dissertation, we present a flexible method for characterizing time-varying behavior during decision-making experiments. Our method consists of a dynamic logistic regression model, parametrized by a set of time-varying weights that express dependence on sensory stimuli as well as task-irrelevant covariates, such as stimulus, choice, and answer history. PsyTrack, the open-source implementation of our method, allows us to characterize behavior on tens of thousands of trials within minutes. We apply our model to behavioral data collected from mice, rats, and human subjects learning either a visual detection or an auditory discrimination task. We uncover the detailed evolution of an animal’s strategy during learning, including adaptation to time-varying task statistics, suppression of sub-optimal strategies, and shared behavioral dynamics between subjects within an experimental population.

Finally, we extend our model to not only track changes in behavior, but also to infer the learning rules animals use to drive those changes. With the ability to decompose a strategy into a learning component and a noise component, we are able to compare and describe the different learning rules that animals may be using to update their policy. The ability to quantify complex and dynamic behavior at a trial-by-trial resolution enables exciting opportunities for animal training and opens a path toward a more rigorous understanding of the behavioral dynamics at play as animals learn.