The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.18)
pp: 2104-2113
Nafees Ahmed , Stony Brook University
Ziyi Zheng , Stony Brook University
Klaus Mueller , Stony Brook University
ABSTRACT
Due to the inherent characteristics of the visualization process, most of the problems in this field have strong ties with human cognition and perception. This makes the human brain and sensory system the only truly appropriate evaluation platform for evaluating and fine-tuning a new visualization method or paradigm. However, getting humans to volunteer for these purposes has always been a significant obstacle, and thus this phase of the development process has traditionally formed a bottleneck, slowing down progress in visualization research. We propose to take advantage of the newly emerging field of Human Computation (HC) to overcome these challenges. HC promotes the idea that rather than considering humans as users of the computational system, they can be made part of a hybrid computational loop consisting of traditional computation resources and the human brain and sensory system. This approach is particularly successful in cases where part of the computational problem is considered intractable using known computer algorithms but is trivial to common sense human knowledge. In this paper, we focus on HC from the perspective of solving visualization problems and also outline a framework by which humans can be easily seduced to volunteer their HC resources. We introduce a purpose-driven game titled “Disguise” which serves as a prototypical example for how the evaluation of visualization algorithms can be mapped into a fun and addicting activity, allowing this task to be accomplished in an extensive yet cost effective way. Finally, we sketch out a framework that transcends from the pure evaluation of existing visualization methods to the design of a new one.
INDEX TERMS
Decision support systems, Handheld computers, Human factors, Color, Games, Computation theory, color blending, Human computation, perception, evaluation
CITATION
Nafees Ahmed, Ziyi Zheng, Klaus Mueller, "Human Computation in Visualization: Using Purpose Driven Games for Robust Evaluation of Visualization Algorithms", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2104-2113, Dec. 2012, doi:10.1109/TVCG.2012.234
REFERENCES
[1] (March 2012). ASP.NET AJAX. Available: http://www.asp.netajax
[2] March 2012). Facebook social plugin. Available: http://developers.facebook.com/docsplugins /
[3] (March 2012). GLSL. Available: http://www.openg1.orgdocumentationlglsl/
[4] (March 2012). Visualization Pipeline. Available: http://www.infovis-wiki.net/index.phpVisualization Pipeline
[5] (March 2012). WebGL. Available: http://www.khronos.orgwebgl/
[6] L. von Ahn, B. Maurer, C. McMillen., D. Abraham, and M. Blum,“recaptcha: Human-based character recognition via web security measures”, Science, 321, pp. 1465-1468, 2008.
[7] Luis von Ahn and Laura Dabbish,“Labeling images with a computer game”, presented at the Proceedings of the SIGCHI, Vienna, Austria, pp. 319-326, 2004.
[8] Luis von Ahn,Shiry Ginosar,Mihir Kedia,Ruoran Liu,, and Manuel Blum,“Improving accessibility of the web with a computer game”, presented at the Proceedings of the SIGCHI, Montreal, Quebec, Canada, pp. 79-82, 2006.
[9] Luis von Ahn,Mihir Kedia,, and Manuel Blum,“Verbosity: a game for collecting common-sense facts”, presented at the Proceedings of the SIGCHI, Montreal, Quebec, Canada, pp. 75-78, 2006.
[10] Luis von Ahn,Ruoran Liu,, and Manuel Blum,“Peekaboom: a game for locating objects in images”, presented at the Proceedings of the SIGCHI, Montreal, Quebec, Canada, pp. 55-64, 2006.
[11] Michael S. Bernstein,Greg Little,Robert C. Miller,Bjorn Hartmann,Mark S. Ackerman,David R. Karger,David Crowell,, and Katrina Panovich,“Soylent: a word processor with a crowd inside”, presented at the Proceedings of UIST, New York, New York, USA, pp. 313-322, 2010.
[12] Jeffrey P. Bigham,Chandrika Jayant,Hanjie Ji,Greg Little,Andrew Miller,Robert C. Miller,Robin Miller,Aubrey Tatarowicz,Brandyn White,Samual White,, and Tom Yeh,“VizWiz: nearly real-time answers to visual questions”, presented at the Proceedings of UIST, New York, New York, USA, pp. 333-342, 2010.
[13] Chaomei Chen,“Top 10 Unsolved Information Visualization Problems”, IEEE Comput. Graph. Appl., 25, pp. 12-16, 2005.
[14] Johnson Chuang,Daniel Weiskopf,, and Torsten Moller,“Hue-Preserving Color Blending”, IEEE Transactions on Visualization and Computer Graphics, 15, pp. 1275-1282, 2009.
[15] Forrester Cole,Kevin Sanik,Doug DeCarlo,Adam Finkelstein,Thomas Funkhouser,Szymon Rusinkiewicz,, and Manish Singh,“How Well Do Line Drawings Depict Shape”, ACM Transactions on Graphics (Proc. SIGGRAPH), 28, 2009.
[16] Seth Cooper,Firas Khatib,Adrien Treuille,Janos Barbero,Jeehyung Lee,Michael Beenen,Andrew Leaver-Fay,David Baker,Zoran Popovic,, and Foldit players,“Predicting protein structures with a multiplayer online game”, Nature, 466, pp. 756-760, 2010.
[17] C. Correa and Ma Kwan-Liu,“The Occlusion Spectrum for Volume Classification and Visualization”, Visualization and Computer Graphics, IEEE Transactions on, 15, pp. 1465-1472, 2009.
[18] J. Davis,J. Arderiu, H. Lin, Z. Nevins., S. Schuon, O. Gallo,, and Yang Ming-Hsuan,“The HPU”, in Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 9-16.
[19] M. Franklin, D. Kossmann, T. Kraska., S. Ramesh, and R. Xin,“CrowdDB: Answering queries with crowdsourcing”, presented at the Proceedings of the 2011 ACM SIGMOD. 00.61-72. 2011.
[20] David Alan Grier,When Computers were Human: Princeton University Press. 2005.
[21] R. B. Haber and D. A. McNabb,“Visualization idioms: A conceptual model for scientific visualization systems”, Visualization in Scientific Computing, pp. 74-93, 1990.
[22] Jeffrey Heer and Michael Bostock,“Crowdsourcing graphical perception: using mechanical turk to assess visualization design”, presented at the Proceedings of SIGCHI, Atlanta, Georgia, USA, pp. 203-212, 2010.
[23] Chang Hu,Benjamin B. Bederson,Philip Resnik,, and Yakov Kronrod,“MonoTrans2: a new human computation system to support monolingual translation”, presented at the Proceedings of SIGCHI, Vancouver, BC, Canada, pp. 1133-1136, 2011.
[24] Panagiotis G. Ipeirotis,Foster Provost,, and ling Wang,“Quality management on Amazon Mechanical Turk”, presented at the Proceedings of the ACM SIGKDD Workshop on Human Computation, Washington DC, pp. 64-67, 2010.
[25] Aniket Kittur,Ed H. Chi,, and Bongwon Suh,“Crowdsourcing user studies with Mechanical Turk”, presented at the Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, Florence, Italy, pp. 453-456, 2008.
[26] Robert Kosara and Caroline Ziemkiewicz,“Do Mechanical Turks dream of square pie charts?”, presented at the Proceedings of the 3rd BELIV'10 Workshop: BEyond time and errors: novel evaLuation methods for Information Visualization, Atlanta, Georgia, pp. 63-70, 2010.
[27] Lars Kuhne,Joachim Giesen,Zhiyuan Zhang,Sungsoo Ha,, and Klaus Mueller,“A Data-Driven Approach to Hue-Preserving Color-Blending“, IEEE Transaction on Visualization and Computer Graphics, 2012.
[28] Anand P. Kulkarni,Matthew Can,, and Bjoern Hartmann,“Turkomatic: automatic recursive task and workflow design for mechanical turk”, presented at the Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems, Vancouver, BC, Canada, pp. 2053-2058, 2011.
[29] Edith Law and Luis von Ahn, Human Computation: Morgan & Claypool Publishers, 2011.
[30] Edith Law,Luis Von Ahn,Roger B. Dannenberg,, and Mike Crawford,“TagATune: A Game for Music and Sound Annotation”, ISMIR, pp. 361-364, 2007.
[31] Greg Little,Lydia B. Chilton,Max Goldman,, and Robert C. Miller,“TurKit: tools for iterative tasks on mechanical Turk”, presented at the Proceedings of the ACM SIGKDD Workshop on Human Computation, Paris, France, pp. 29-30, 2009.
[32] Wang Lujin,J. Giesen,K. T. McDonnell, P. Zolliker, and K. Mueller,“Color Design for Illustrative Visualization”, IEEE Transaction on Visualization and Computer Graphics, 14, pp. 1739-1754, 2008.
[33] Adam Marcus,Eugene Wu,David Karger,Samuel Madden,, and Robert Miller,“Human-powered sorts and joins”, Proc. VLDB Endow., 5, pp. 13-24, 2011.
[34] F. Metteli,“The perception of transparency”, Scientific American, 230, pp. 91-98, 1974.
[35] Chan Ming-Yuen,Wu Yingcai,Mak Wai-Ho,Chen Wei,, and Qu Huamin,“Perception-Based Transparency Optimization for Direct Volume Rendering”, Visualization and Computer Graphics, IEEE Transactions on, 15, pp. 1283-1290, 2009.
[36] Catherine Plaisant,“The challenge of information visualization evaluation”, presented at the Proceedings of the working conference on Advanced visual interfaces, Gallipoli, Italy, pp. 109-116, 2004.
[37] Thomas Porter and Tom Duff,“Compositing digital images”, SIGGRAPH Comput. Graph., 18, pp. 253-259, 1984.
[38] Joel Ross,Lilly Irani,M. Six Silberman,Andrew Zaldivar,, and Bill Tomlinson,“Who are the crowdworkers?: shifting demographics in mechanical turk”, presented at the Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, Atlanta, Georgia, USA, pp. 2863-2872, 2010.
[39] Stuart Russel and Peter Norvig,“Intelligent Agents”, in Artificial Intelligence: A Modern Approach, Third Edition ed: Prentice Hall, 2010, p. 54.
[40] Petr Sercda,Anna Vilanova,, and Frans A. Gerritsen,“Automating Transfer Function Design for Volume Rendering Using Hierarchical Clustering of Material Boundaries”, Eurographics/IEEE VGTC Symposium on Visualization (Euro‘Vis), pp. 243-250, 2006.
[41] M. Singh and BL. Anderson,“Perceptual assignment of opacity to translucent surfaces: The role of image blur”, Perception, 31, pp. 531-552, 2002.
[42] M. Singh and BL. Anderson,“Toward a perceptual theory of transparency”, Pschol Rev, 109, pp. 492-519, 2002.
[43] Iris Sprow,Zofia Baranczuk,Tobias Stamm,, and Peter Zolliker,“Web-based psychometric evaluation of image quality”, Image Quality and System Performance VI (Proc. SPIE), 7242, 2009.
[44] D. G. Stork,“The Open Mind Initiative”, IEEE Intelligent Systems & Their Applications, 14, pp. 19-20, 1999.
[45] Jarke J. Van Wijk,“The value of visualization”, IEEE Visualization, pp. 79-86, 2005.
[46] Ziyi Zheng,Nafees Ahmed,, and Klaus Mueller,“iView: A Feature Clustering Framework for Suggesting Informative Views in Volume Visualization”, Visualization and Computer Graphics, IEEE Transactions on, 17, pp. 1959-1968, 2011.
15 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool