Asymmetric Learning Rates for Positive and Negative Feedback: A Formal Model Comparison

Abstract

Reinforcement learning studies how learning systems interact with their environments in order to maximize a numerical reward signal. One of the central concepts in reinforcement learning is the reward “prediction error”, which is the numerical value of the reward received minus the expected reward value. The classic Rescorla-Wagner model posits that the learning agent updates their original reward estimate by stepwise error-correction: multiplying the prediction error by a learning rate parameter. The main limitation with the Rescorla-Wagner model is the implied valence symmetry with which feedback updates an action’s value estimate. This contradicts evidence that learning from positive and negative feedback has different effects on behavior and decision-making. In this thesis, I conduct a formal model comparison of the Rescorla-Wagner model with an alternative class of asymmetric learning models which discriminate based on valence. Our analyses show that behavioral choice data in a probabilistic learning experiment is more accurately described by an asymmetric learning algorithm rather than a symmetric learning rule which does not discriminate based on valence.