The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.18)
pp: 2441-2448
Heike Hofmann , Iowa State University
Lendie Follett , Iowa State University
Mahbubul Majumder , Iowa State University
Dianne Cook , Iowa State University
ABSTRACT
Lineups [4, 28] have been established as tools for visual testing similar to standard statistical inference tests, allowing us to evaluate the validity of graphical findings in an objective manner. In simulation studies [12] lineups have been shown as being efficient: the power of visual tests is comparable to classical tests while being much less stringent in terms of distributional assumptions made. This makes lineups versatile, yet powerful, tools in situations where conditions for regular statistical tests are not or cannot be met. In this paper we introduce lineups as a tool for evaluating the power of competing graphical designs. We highlight some of the theoretical properties and then show results from two studies evaluating competing designs: both studies are designed to go to the limits of our perceptual abilities to highlight differences between designs. We use both accuracy and speed of evaluation as measures of a successful design. The first study compares the choice of coordinate system: polar versus cartesian coordinates. The results show strong support in favor of cartesian coordinates in finding fast and accurate answers to spotting patterns. The second study is aimed at finding shift differences between distributions. Both studies are motivated by data problems that we have recently encountered, and explore using simulated data to evaluate the plot designs under controlled conditions. Amazon Mechanical Turk (MTurk) is used to conduct the studies. The lineups provide an effective mechanism for objectively evaluating plot designs.
INDEX TERMS
Observers, Accuracy, Data models, Visual analytics, Statistical analysis, Inference mechanisms, Efficiency of displays, Lineups, Visual inference, Power comparison
CITATION
Heike Hofmann, Lendie Follett, Mahbubul Majumder, Dianne Cook, "Graphical Tests for Power Comparison of Competing Designs", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2441-2448, Dec. 2012, doi:10.1109/TVCG.2012.230
REFERENCES
[1] A. Agresti and B. A Coull., Approximate is better than “exact” for interval estimation of binomial proportions The American Statistician, 52(2): 119-126, 1998.
[2] D. Bates, M. Maechler, and B. Bolker., lme4: Linear mixed-effects models using S4 classes, 2011. R package version 0.999375– 42.
[3] Y. Benjamini and Y. Hochberg, Controlling the false discov-ery rate: a practical and powerful approach to multiple testing Journal of the Royal Statistical Society Series B, 57(1): 289-300, 1995.
[4] A. Buja, D. Cook, H. Hofmann, M. Lawrence, E.-K. Lee,D. F. Swayne,, and H. Wickham., Statistical inference for exploratory data analysis and model diagnostics. Royal Society Philosophical Transactions A, 367(1906): 4361-4383, 2009.
[5] W. S, Cleveland Elements of graphing data. Hobart Press, 1994.
[6] W. S Cleveland and R. McGill., Graphical perception: Theory, experimentation, and application to the development of graphical methods Journal of the American Statistical Association, 79(387): PP. 531-554, 1984.
[7] G. Ellis and A. Dix., An explorative analysis of user evaluation studies in information visualisation. In Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization, BELIV ‘06, pages 1-7, New York, NY, USA, 2006. ACM.
[8] J. Heer and M. Bostock., Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In Proceedings of the 28th international conference on Human factors in computing systems, CHI fo, pages 203-212, New York, NY, USA, 2010. ACM.
[9] T. Hothorn, F. Bretz, and P. Westfall, Simultaneous inference in general parametric models Biometrical Journal, 50(3): 346-363, 2008.
[10] R. Kosara and C. Ziemkiewicz., Do mechanical turks dream of square pie charts? In Proceedings BEyond time and errors: novel evaLuation methods for Information Visualization (BE-LIV), pages 373-382. ACM Press, 2010.
[11] S. Kosslyn., Graph designfor the eye and mind. Oxford Univer-sity Press, 2006.
[12] M. Majumder, H. Hofmann, and D. Cook., Visual statistical inference for regression parameters. Technical Report 13, Iowa State University, Department of Statistics, 2011.
[13] M. Majumder, H. Hofmann, and D. Cook., Validation of visual statistical inference, with applications to linear models. Technical Report 4, Iowa State University, Department of Statistics, 2012.
[14] R. Mazza and A. Berre., Focus group methodology for evaluating information visualization techniques and tools. In Information Visualization, 2007. IV ‘07. 11th International Conference, pages 74-80, Los Alamitos, CA, USA, 2007. IEEE Computer Society.
[15] T. Munzner, A nested model for visualization design and validation Visualization and Computer Graphics, IEEE Transactions on, 15(6): 921-928, nov.-dec. 2009.
[16] NOAA Climatic Data Center. Past weather. httP://weather. noaa. gov/weather/current ,last accessed August 15, 2012, 2011.
[17] C. North, Toward measuring visualization insight IEEE Computer Graphics and Applications, pages 6-9, 2006.
[18] H.-P. Piepho, An algorithm for a letter-based representation of all-pairwise comparisons Journal of Computational and Graphical Statistics, 13(2): 456-466, 2004.
[19] J. Pinheiro and D. Bates., Mixed-effects models in Sand S-Plus. Springer, 2000.
[20] C. Plaisant., The challenge of information visualization evaluation. In Proceedings of Conference on Advanced Visual Inter-faces, pages 109-116, New York, NY, USA, 2004. ACM.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool