Information Sampling, Learning and Exploration

Abstract

Our information-rich world often presents us with many available choices, a lot of uncertainty, and noisy feedback that we must make sense of. To make good decisions, we must extract information from our environments, form accurate representations of the available options, and perform efficient computations that help us choose the most goal-relevant actions. This thesis investigated how humans learn information from the tasks they perform, and how they use that information to estimate the values of their actions, and adaptively adjust their behavior.

Chapter 2 compared two conceptually different models of human learning strategies in a probabilistic learning task, finding that humans are not Bayes-optimal when extracting the value of relevant features in noisy environments, and that it was possible to directly influence their performance by tailoring the information they received to their individual learning strategies. Delving further into the question of how people learn information about available options, Chapter 3 introduced the exploration-exploitation dilemma. Two experiments – one using a two-armed bandit task with a decision-horizon manipulation, the other using a similar wheel-of-fortune design with an additional risk manipulation – suggested that people use exploration as a mechanism for acquiring information about unknown options, and that exploration strategies are affected by the decision horizon, risk and ambiguity. Chapter 4 examined the connection between information-seeking and exploration in terms of its effects on motivation. Five experiments showed that participants’ boredom ratings depended on task informativeness, as well as on the perceived opportunity cost of performing a task, and that higher boredom correlated with increased exploration. A normative model was proposed, accounting for adaptive, boredom-driven exploration in environments in which locally maximizing reward must be balanced with the need for learning useful information about the global environment structure.

Overall, this dissertation investigated the relationship between information, learning and exploration, and it determined key factors that drive exploratory behavior in uncertain decision contexts, as well as the potential role of exploration as an adaptive information-sampling strategy.