Interpreting Graphical Representations of Data: Understanding Biases in the Perception of Data Visualizations

Abstract

The rapid proliferation of data has made the graphical display of quantitative information indispensable for conveying information in a concise and compelling manner. Despite our reliance on data visualization, we do not fully understand what people see when they look at a graph. Do our intuitive impressions of visualized data align with its statistical properties? Study 1 (n = 360) asked participants to judge which of two distributions seemed higher overall when presented side-by-side. The results of Study 1 found that people were far less sensitive to differences between distributions than our scientific standards would suggest, and that differences in central tendency seem to matter more than differences in variance. Study 2 (n = 52) sought to determine whether participants' relatively poor discrimination comparisons in Study 1 were indicative of a visual insensitivity to statistical properties, or difficulty in making comparative judgments. To do this, distributions were presented both in pairs and in isolation, and participants were asked to identify basic properties of the dataset (mean and variance). The results of Study 2 suggested that low accuracy in discriminating between distributions might be the result of a failure to make comparisons between two distributions, rather than extract accurate summary statistics. These findings are of especial interest for policymakers, who are increasingly reliant on data visualizations both as a decision aid and as a tool to communicate decisions to the public. Since these findings seem to contradict our previous notions about how the public interprets data visualization, we must reconsider both the design and presentation of data to the public.