Spectral methods and MLE: a modern statistical perspective

Abstract

Modern statistical analysis often requires the integration of statistical thinking and algorithmic thinking. There are new challenges posed for classical estimation principles. Indeed, in high-dimensional problems, statistically sound estimation procedures such as maximum-likelihood estimation (MLE) may be difficult to compute, at least in the naive form. Also, spectral methods such as principal component analysis, which enjoy low computational costs, have unclear statistical guarantees in general.

This thesis addresses both spectral methods and MLE in a wide range of estimation problems, including high-dimensional factor models, community detection, matrix completion, synchronization problems, etc. The fundamental structure that underlies these problems is low rank, which is a core structure in modern statistics and machine learning. The low rank structure enables the use of spectral methods, and it allows efficient algorithms for solving nonconvex optimization problems with certain structural assumptions.

The contribution of this thesis includes the following. It reveals interesting phenomena about entrywise behavior of eigenvectors, leading to sharp `∞ perturbation bounds. These bounds are provided in both the deterministic regime and the random regime. Besides, a stability-based strategy, namely leave-one-out, is proposed to analyze nonconvex optimization problems. Finally, a moments-based spectral aggregation method is proposed to handle practical issues such as data heterogeneity.