The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.18)
pp: 2536-2545
ABSTRACT
People have difficulty understanding statistical information and are unaware of their wrong judgments, particularly in Bayesian reasoning. Psychology studies suggest that the way Bayesian problems are represented can impact comprehension, but few visual designs have been evaluated and only populations with a specific background have been involved. In this study, a textual and six visual representations for three classic problems were compared using a diverse subject pool through crowdsourcing. Visualizations included area-proportional Euler diagrams, glyph representations, and hybrid diagrams combining both. Our study failed to replicate previous findings in that subjects’ accuracy was remarkably lower and visualizations exhibited no measurable benefit. A second experiment confirmed that simply adding a visualization to a textual Bayesian problem is of little help, even when the text refers to the visualization, but suggests that visualizations are more effective when the text is given without numerical values. We discuss our findings and the need for more such experiments to be carried out on heterogeneous populations of non-experts.
INDEX TERMS
Visualization, Bayesian methods, Sociology, Statistical analysis, Crowdsourcing, Breast cancer, crowdsourcing, Bayesian reasoning, base rate fallacy, probabilistic judgment, Euler diagrams, glyphs
CITATION
Luana Micallef, Pierre Dragicevic, Jean-Daniel Fekete, "Assessing the Effect of Visualizations on Bayesian Reasoning through Crowdsourcing", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2536-2545, Dec. 2012, doi:10.1109/TVCG.2012.199
REFERENCES
[1] I. Ajzen, Intuitive theories of events and the effects of base-rate information on prediction Personality and Social Psychology, 35(5): 303-314, 1977.
[2] J. S. Ancker,E. U. Weber,, and R. Kukafka., Effect of arrangement of stick figures on estimates of proportion in risk graphics Medical Decision Making, 31(1): 143-150, 2011.
[3] M. Bar-Hillel., The base-rate fallacy in probability judgments. Acta Psy-chologica, 44(3): 211-233, 1980.
[4] A. K. Barbey and S. A. Sloman., Base-rate respect: From ecological rationality to dual processes Behavioral and Brain Sciences, 30(3): 241254; discussion 255–297, 2007.
[5] A. K. Barbey and S. A. Sloman., Base-rate respect: From statistical formats to cognitive structures Behavioral and Brain Sciences, 30(3): 287-297, 2007.
[6] M. I. Bauer and P. N., Johnson-Laird. How Diagrams Can Improve Reasoning Psychological Science, 4(6): 372-378, 1993.
[7] F. Benoy and P. Rodgers., Evaluating the Comprehension of Euler Diagrams. In Conf Information Visualization (IV), pages 771-778, 2007.
[8] G. L. Brase., Pictorial representations in statistical reasoning Applied Cognitive Psychology, 23(3): 369-381, 2009.
[9] S. Brown, J. Culver, K. Osann., D. MacDonald, S. Sand., A. Thornton, M. Grant., D. Bowen, K. Metcalfe., H. Burke, M. Robson., S. Friedman, and J. Weitzel, Health literacy, numeracy, and interpretation of graphical breast cancer risk estimates Patient Education and Counseling, 83(1): 92-98, 2011.
[10] K. Burns., Painting pictures to augment advice. In Conf Advanced Visual Interfaces, pages 344-349, 2004.
[11] G. B. Chapman and J. Liu., Numeracy, frequency, and Bayesian reasoning Judgment and Decision Making, 4(1): 34-40, 2009.
[12] J. J. Chen,N. J. Menezes,A. D. Bradley,, and T. A. North., Opportuni-ties for Crowdsourcing Research on Amazon Mechanical Turk Human Factors, 5: 3, 2011.
[13] Chicago Tribune. A false hiv test caused 18 months of hell, (3/5/1993).
[14] J. Cohen, The Earth is Round (p <. 05) American Psychologist, 49(12): 997-1003, 1994.
[15] W. G. Cole., Understanding Bayesian reasoning via graphical displays. In ACM Conf Human Factors in Computing Systems, pages 381-386, 1989.
[16] W. G. Cole and J. E. Davidson., Graphic Representation Can Lead To Fast and Accurate Bayesian Reasoning. In Symp Computer Application in Medical Care, pages 227-231, 1989.
[17] C. Collins, G. Penn, and S. Carpendale, Bubble sets: revealing set relations with isocontours over existing visualizations IEEE Transactions on Visualization and Computer Graphics, 15(6): 1009-1016, 2009.
[18] L. Cosmides and J. Tooby, Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty Cognition, 58(1), 1996.
[19] T. C. Davis,N. C. Dolan,M. R. Ferreira, C. Tomori, K. W. Green,A. M. Sipler,, and C. L. Bennett., The role of inadequate health literacy skills in colorectal cancer screening Cancer Investigation, 19(2): 193-200, 2001.
[20] J. G. Dolan and S. Iadarola., Risk communication formats for low proba-bilityevents: an exploratory study of patient preferences BMC Medical Informatics and Decision Making, 8(14): 14, 2008.
[21] D. M, Eddy Probabilistic reasoning in clinical medicine: problems and opportunities, pages 249-267. Cambridge University Press, 1982.
[22] R. B. Ekstrom,J. W. French,H. H. Harman,, and D. Dermen, Manualfor Kit of Factor-Referenced Cognitive Tests. Educational Testing Service, 1976.
[23] A. Fagerlin,B. J. Zikmund-Fisher,P. A. Ubel, A. Jankovic, H. A. Derry,, and D. M. Smith., Measuring numeracy without a math test: development of the Subjective Numeracy Scale. Medical Decision Making, 27(5): 67280, 2007.
[24] M. Galesic,R. Garcia-Retamero,, and G. Gigerenzer., Using Icon Arrays to Communicate Medical Risks: Overcoming Low Numeracy. Health Psychology, 28(2): 210-216, 2009.
[25] G. Gigerenzer., Ecological intelligence: An adaptation for frequencies, 39, pages 9-29. Oxford University Press, 1998.
[26] G. Gigerenzer, W. Gaissmaier, E. Kurz-Milcke,L. M. Schwartz,, and S. Woloshin., Helping Doctors and Patients Make Sense of Health Statistics. Psychological Science in the Public Interest, 8(2): 53-96, 2007.
[27] G. Gigerenzer and U. Hoffrage, How to Improve Bayesian Reasoning Without Instruction: Frequency Formats Psychological Review, 102(4): 684-704, 1995.
[28] V. Girotto and M. Gonzalez., Solving probabilistic and statistical problems: a matter of information structure and question form. Cognition, 78(3): 247-276, 2001.
[29] I. J. Good., When batterer turns murderer Nature, 375(6532): 541, 1995.
[30] J. Heer and M. Bostock., Crowdsourcing Graphical Perception: Using Mechanical Turk to Assess Visualization Design. In ACM Conf Human Factors in Computing Systems, pages 203-212, 2010.
[31] U. Hoffrage and G. Gigerenzer, How to Improve the Diagnostic Inferences of Medical Experts Experts in Science and Society, pages 249268, 2004.
[32] U. Hoffrage, S. Lindsey, R. Hertwig,, and G. Gigerenzer., Medicine: Communicating statistical information. Science, 290(5500): 2261-2262, 2000.
[33] P. N. Johnson-Laird, P. Legrenzi, V. Girotto,M. S. Legrenzi,, and J. P. Cavemi., Naive probability: a mental model theory of extensional reasoning Psychological Review, 106(1): 62-88, 1999.
[34] D. Kahneman, P. Slovic, and A. Tversky., Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, 1982.
[35] V. Kellen, S. Chan, and X. Fang., Facilitating conditional probability problems with visuals. In Conf Human-Computer Interaction, pages 63-71, 2007.
[36] J. J. Koehler., One in Millions, Billions and Trillions: Lessons from People V. Collins (1968) for People V. Simpson (1995) Legal Education, 47: 214-223, 1997.
[37] K. Koffka., Principles of Gestalt Psychology. Harcourt, Brace, 1935.
[38] L. Micallef and P. Rodgers., Drawing Area-Proportional Venn-3 Diagrams Using Ellipses. Technical Report TR-3–11, School of Computing, University of Kent, UK, 2011. http://www.eulerdiagrams.orgeulerAPE.
[39] A. L. Nichols and J. K. Maner., The good-subject effect: investigating participant demand characteristics General Psychology, 135(2): 151-165, 2008.
[40] G. Paolacci, J. Chandler, and P. G. Ipeirotis., Running experiments on Amazon Mechanical Turk Judgment and Decision Making, 5(5): 411-419, 2010.
[41] E. C. Poulton and P. R. Freeman., Unwanted asymmetrical transfer effects with balanced experimental designs Psychological Bulletin, 66(1): 1-8, 1966.
[42] M. Price, R. Cameron, and P. Butow, Communicating risk information: the influence of graphical display format on quantitative information perception-accuracy, comprehension and preferences Patient Education and Counseling, 69(1-3): 121-128, 2007.
[43] N. H. Riche and T. Dwyer., Untangling Euler Diagrams IEEE Transactions on Visualization and Computer Graphics, 16(6): 1090-1099, 2010.
[44] P. Rodgers, L. Zhang, and A. Fish., General Euler Diagram Generation. In Conf Diagrams, LNCS (LNAI) 5223, pages 13-27, 2008.
[45] P. Rodgers, L. Zhang, and H. Purchase., Wellformedness Properties in Euler Diagrams: Which Should Be Used? IEEE Transactions on Visualization and Computer Graphics, 18(7): 1089-1100, 2012.
[46] J. Ross, L. Irani, M. S. Silberman, A. Zaldivar, and B. Tomlinson., Who are the Crowdworkers? Shifting Demographics in Mechanical Turk. In ACM Conf Human Factors in Computing Systems, pages 2863-2872, 2010.
[47] T. A. Salthouse., The processing-speed theory of adult age differences in cognition Psychological Review, 103(3): 403-428, 1996.
[48] P. Sedlmeier and G. Gigerenzer, Teaching Bayesian reasoning in less than two hours Experimental Psychology General, 130(3): 380-400, 2001.
[49] S. A. Sloman, D. Over, L. Slovak,, and J. M. Stibel., Frequency illusions and other fallacies Organizational Behavior and Human Decision Processes, 91(2): 296-309, 2003.
[50] G. Stine., Acquired immune deficiency syndrome: biological, medical, social, and legal issues. Prentice Hall, 1996.
[51] E. R. Stone,J. F. Yates,, and A. M. Parker., Effects of numerical and graphical displays on professed risk-taking behavior Experimental Psychology Applied, 3(4), 1997.
[52] J. Surowiecki., The Wisdom of Crowds. Anchor, 2005.
[53] E. R, Tufte Beautiful Evidence. Graphics Press, 2006.
[54] B. Victor. Explorable Explanations. http:/worrydream.com. 2011.
[55] H. G. Welch and W. C. Black., Overdiagnosis in Cancer J National Cancer Institute, 102(9): 605-613, 2010.
[56] L. Wilkinson, Exact and Approximate Areaproportional Circular Venn and Euler Diagrams IEEE Transactions on Visualization and Computer Graphics, 18(2): 321-331, 2011.
[57] K. Yamagishi., Facilitating normative judgments of conditional probability: frequency or nested sets? Experimental Psychology, 50(2): 97-106, 2003.
[58] P.-H. Zahl and J. Maehlen, Overdiagnosis in mammography screening Tidsskr Norske Laege, 124(17): 2238-2239, 2004.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool