The Community for Technology Leaders
RSS Icon
Issue No.12 - Dec. (2012 vol.18)
pp: 2739-2748
Andrew Vande Moere , KU Leuven
Martin Tomitsch , University of Sydney
This paper reports on a between-subject, comparative online study of three information visualization demonstrators that each displayed the same dataset by way of an identical scatterplot technique, yet were different in style in terms of visual and interactive embellishment. We validated stylistic adherence and integrity through a separate experiment in which a small cohort of participants assigned our three demonstrators to predefined groups of stylistic examples, after which they described the styles with their own words. From the online study, we discovered significant differences in how participants execute specific interaction operations, and the types of insights that followed from them. However, in spite of significant differences in apparent usability, enjoyability and usefulness between the style demonstrators, no variation was found on the self-reported depth, expert-rated depth, confidence or difficulty of the resulting insights. Three different methods of insight analysis have been applied, revealing how style impacts the creation of insights, ranging from higher-level pattern seeking to a more reflective and interpretative engagement with content, which is what underlies the patterns. As this study only forms the first step in determining how the impact of style in information visualization could be best evaluated, we propose several guidelines and tips on how to gather, compare and categorize insights through an online evaluation study, particularly in terms of analyzing the concise, yet wide variety of insights and observations in a trustworthy and reproducable manner.
Visualization, Subspace constraints, Data visualization, Abstracts, Usability, Benchmark testing, Electronic mail, user experience, Visualization, design, style, aesthetics, evaluation, online study
Andrew Vande Moere, Martin Tomitsch, Christoph Wimmer, Boesch Christoph, Thomas Grechenig, "Evaluating the Effect of Style in Information Visualization", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2739-2748, Dec. 2012, doi:10.1109/TVCG.2012.221
[1] S. Bateman, R. Mandryk, C. Gutwin., A. Genest, D. McDine, and C. Brooks,“Useful Junk? The Effects of Visual Embellishment on Comprehension and Memorability of Charts,” Conference on Human Factors in Computing Systems (CHI‘10), ACM, 2010, pp. 2573-2582.
[2] Bestiario, “reMap: a project by Bestiario based on,” 2009;
[3] N. Cawthon and A. Vande Moere,“The Effect of Aesthetic on the Usability of Data Visualization,” IEEE Conference on Information Visualization (IV‘07), IEEE, 2007, pp. 637-648.
[4] R. Chang,C. Ziemkiewicz, and T.M.R. Green,W., , “Defining Insight for Visual Analytics,” IEEE Computer Graphics and Applications 29, no. 2, 2009, pp. 14-17.
[5] Y. Chen,J. Yang, and W. Ribarsky,“Toward Effective Insight Management in Visual Analytics Systems,” IEEE Pacific Visualization Symposium (Pacific Vis ‘09), IEEE, 2009, pp. 49-56.
[6] C. Collins,G. Penn, and S. Carpendale,“Bubble Sets: Revealing Set Relations with Isocontours over Existing Visualizations, ” IEEE Transactions on Visualization and Computer Graphics, 15, 6, 2009, pp. l009-1016.
[7] Stamen Design,“Digg Labs,” 2007; http:/
[8] N. Diakopoulos,F. Kivran-Swaine, and M. Naaman,“Playable Data: Characterizing the Design Space of Game-y Infographics,” Conference on Human Factors in Computing Systems (CHI‘11 ACM, 2011.
[9] J. Harris and S. Kamvar,“We Feel Fine - An Exploration of Human Emotion, in Six Movements,” 2006;
[10] P. Isenberg, T. Zuk, C. Collins, and S. Carpendale,“Grounded Evaluation of Information Visualizations,” Conference on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV ‘08), ACM, 2008.
[11] M. Jern,“Collaborative Web-Enabled Geoanalytics Applied to OECD Regional Data,” Cooperative Design, Visualization, and Engineering (CDVE‘09)5738/2009, Springer, 2009, pp. 379-394.
[12] R. Kosara,“Visualization Criticism - The Missing Link Between Information Visualization and Art,” Conference on Information Visualization (IV‘07), IEEE, 2007, pp. 631-636.
[13] M. Kurosu and K. Kashimura,“Apparent Usability vs. Inherent Usability: Experimental Analysis on the Determinants of the Apparent Usability,” Conference on Human factors in Computing Systems (CHI‘95), ACM, 1995, pp. 292-293.
[14] UC Berkeley Visualization Lab, “Flare - Data Visualization for the Web,” 2008;
[15] A. Lau and A. Vande, Moere, “Towards a Model of Information Aesthetic Visualization,” International Conference on Information Visualisation (IV‘07), IEEE, 2007, pp. 87-92.
[16] B. Mueller,“Poetry on the Road,” 2004; .
[17] D. Norman,“Emotion & Design: Attractive Things Work Better,” interactions, 9, 4, 2002, pp. 36-42.
[18] C. North,“Toward Measuring Visualization Insight,” IEEE Computer Graphics, 26, 3, 2006, pp. 6-9.
[19] C. North,P. Saraiya, and K. Duca,“A Comparison of Benchmark Task and Insight Evaluation Methods for Information Visualization,” Information Visualization, 10, 3, 2011, pp. 162-181.
[20] V. Postrel,The Substance of Style: How the Rise of Aesthetic Value is Remaking Commerce, Culture and Consciousness, Harper Collins, 2003.
[21] Z. Pousman,J. Stasko, and M. Mateas,“Casual Information Visualization: Depictions of Data in Everyday Life,” IEEE Transactions on Visualization and Computer Graphics, 13, 6, 2007, pp. 1145-1152.
[22] B. Reavis,“Bitalizer - Bending Bits into Structure,” 2008; .
[23] M. Rester, M. Pohl, S. Wiltner., K. Hinum, S. Miksch,C. Popow, and S. Ohmann,“Evaluating an Infovis Technique using Insight Reports,” International Conference on Information Visualization (IV‘07), IEEE, 2007, pp. 693-700.
[24] C. Riekoff,“Texone Code Tree,” 2005; http:/
[25] H. Rosling,“Visual Technology Unveils the Beauty of Statistics and Swaps Policy from Dissemination to Access,” Journal of the International Association for Official Statistics, 24, 1-2, 2007, pp. 103-104.
[26] C. Salimun, H. Purchase, D.R. Simmons, and S. Brewster,“The Effect of Aesthetically Pleasing Composition on Visual Search Performance,” Nordic Conference on Human-Computer Interaction (Nordi CHI’10), ACM, 2010, pp. 422-431.
[27] The New York Times, “Times Developer Network,” 2009; .
[28] N. Tractinsky,“Towards the Study of Aesthetics in Information Technology,” Conference on Information Systems, 2004, pp. 771-780.
[29] N. Tractinsky, and S.-K. A. D. Ikar,“What is Beautiful is Usable” Interacting with Computers, 13, 2, 2000, pp. 127-145.
[30] N. Tractinsky and J. Meyer,“Chart junk or Goldgraph? Effects of Presentation Objectives and Content Desirability on Information Presentation,” MIS Quarterly, 23, 3, 1999, pp. 397-420.
[31] F. Viégas, M. Wattenberg, F. van Ham,J. Kriss, and M. McKeon,“Many Eyes: a Site for Visualization at Internet Scale,” IEEE Transactions on Visualization and Computer Graphics, 13, 6, 2007, pp. 1121-1128.
[32] F. Viégas, and M. Wattenberg,“Artistic Data Visualization: Beyond Visual Analytic,” Lecture Notes in Computer Science, 4564, 15, 2007, pp. 182-191.
[33] J. S. Yi, Y.-a. Kang, J.T. Stasko, and J.A. Jacko,“Understanding and Characterizing Insights: How Do People Gain Insights using Information Visualization?,“ Conference on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV ‘08), ACM, 2008, pp. 1-6.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool