Limited Feedback Models in Online Control

Abstract

The study of online control of linear dynamical systems has attracted increasing interests within the machine learning community. This framework generalizes the classical control theory by relaxing assumptions on the cost and perturbation models. However, even under these relaxed assumptions, full information feedback remains impractical in many settings. This thesis addresses this limitation by exploring online control models with restricted feedback, with a focus on the bandit feedback setting. We contribute efficient algorithms for bandit convex optimization that achieve tight regret guarantees. Subsequently, these results lead to development of novel bandit controller algorithms that push toward optimal rates under minimal assumptions on the cost function class and noise model. Along the way, we also present results in controlling marginally stable systems and offer contributions to other problems in online learning, such as uncertainty quantification in matrix completion and memory- efficient covariance sketching.