Little is known about the cognitive abilities which influence the comprehension of scientific and information visualizations and what properties of the visualization affect comprehension. Our goal in this paper is to understand what makes visualizations difficult. We address this goal by examining the spatial ability differences in a diverse population selected for spatial ability variance. For example, how is spatial ability related to visualization comprehension? What makes a particular visualization difficult or time intensive for specific groups of subjects? In this paper, we present the results of an experiment designed to answer these questions. Fifty-six subjects were tested on a basic visualization task and given standard paper tests of spatial abilities. An equal number of males and females were recruited in this study in order to increase spatial ability variance. Our results show that high spatial ability is correlated with accuracy on our three-dimensional visualization test, but not with time. High spatial ability subjects also had less difficulty with object complexity and the hidden properties of an object.