Knowledge gradient for expensive locally quadratic functions and stochastic optimization of aid allocation

Abstract

In this dissertation, we study the behavior of value of information policies in the presence of a locally quadratic belief model. We show that the well-known behavior of many learning policies for look-up table belief model no longer applies when the underlying truth is parametric. We characterize the behavior of the knowledge gradient policy---a policy that maximizes the one-step value of information in the presence of a parametric belief model. We exploit this insight to derive a simple heuristic rule which we demonstrably show performs comparably to the knowledge gradient policy. The second contribution we make is extend this to a setting where the underlying function is not parametric but rather only locally quadratic. Classical response surface methods sample what it believed to be the optimum, but these experiments tend to have low value of information. By contrast, applying the knowledge gradient when we assume that the true function is quadratic (even though it is not) tends to encourage sampling points close to boundaries, but this ignores the reality that there tends to be very high bias far from the estimated optimal. We assume that there is a bias between the true function

and the quadratic approximation that is Lipschitz continuous. When we imbed this in our belief as a form of uncertainty, distinct from experimental noise, the result is a policy that encourages sampling away from the estimated optimal, but not too far away (this depends on the Lipschitz constant). The final part of this dissertation is an application where we study the allocation of official developmental assistance among recipient countries. We extend the work done in an influential economics study by Collier & Dollar (2002)---which modeled the world in a static framework|to one that models the world over time under uncertainty. We propose a competing backward approximate dynamic programming policy, which considers the effect of immediate decisions on subsequent years, and compare its performance to the myopic policy of Collier & Dollar (2002).