The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.18)
pp: 2859-2868
ABSTRACT
Visual Analytics is “the science of analytical reasoning facilitated by visual interactive interfaces” [70]. The goal of this field is to develop tools and methodologies for approaching problems whose size and complexity render them intractable without the close coupling of both human and machine analysis. Researchers have explored this coupling in many venues: VAST, Vis, InfoVis, CHI, KDD, IUI, and more. While there have been myriad promising examples of human-computer collaboration, there exists no common language for comparing systems or describing the benefits afforded by designing for such collaboration. We argue that this area would benefit significantly from consensus about the design attributes that define and distinguish existing techniques. In this work, we have reviewed 1,271 papers from many of the top-ranking conferences in visual analytics, human-computer interaction, and visualization. From these, we have identified 49 papers that are representative of the study of human-computer collaborative problem-solving, and provide a thorough overview of the current state-of-the-art. Our analysis has uncovered key patterns of design hinging on humanand machine-intelligence affordances, and also indicates unexplored avenues in the study of this area. The results of this analysis provide a common framework for understanding these seemingly disparate branches of inquiry, which we hope will motivate future work in the field.
INDEX TERMS
human computer interaction, data visualisation, groupware, visual interactive interface, affordance-based framework, human computation, human-computer collaboration, visual analytics, analytical reasoning, Human factors, Computation theory, Visual analytics, Resource management, Cognition, framework, Human computation, human complexity, theory
CITATION
R. J. Crouser, R. Chang, "An Affordance-Based Framework for Human Computation and Human-Computer Collaboration", IEEE Transactions on Visualization & Computer Graphics, vol.18, no. 12, pp. 2859-2868, Dec. 2012, doi:10.1109/TVCG.2012.195
REFERENCES
[1] E. Adar., Why i hate mechanical turk research (and workshops). In Proc. of the 29th SIGCHI Conf. on Human factors in computing systems: Workshop on Crowrdsourcing and Human Comnutation. ACM, 2011.
[2] T. Amabile., Creativity in context: Update to “the social psychology of creativity. Westview press, 1996.
[3] A. Anand, L. Wilkinson, and T. Dang., Using random projections to identify class-separating variables in high-dimensional spaces. In Visual Analytics Science and Technology (VAST), Conf. on, pages 263-264. IEEE, 2011.
[4] G. Andrienko, N. Andrienko, S. Rinzivillo., M. Nanni, D. Pedreschi,, and F. Giannotti., Interactive visual clustering of large collections of trajectories. In Visual Analytics Science and Technology (VAST) 2009, Symposium on, pages 3-10. IEEE, 2009.
[5] L. Barrington,D. O’Malley, D. Turnbull, and G. Lanckriet., User-centered design of a social game to tag music. In Proc. of the ACM SIGKDD Workshop on Human Computation; pages 7-10. ACM, 2009.
[6] M. Bell, S. Reeves, B. Brown., S. Sherwood, D. MacMillan., J. Ferguson, and M. Chalmers., Eyespy: supporting navigation through play. In Proc. of the 27th SIGCHI Conf. on Humanfactors in computing systems, pages 123-132. ACM, 2009.
[7] B. Berger and T. Leighton., Protein folding in the hydrophobic-hydrophilic (hp) model is np-complete. Journal of Computational Biology, 5(1): 27-40, 1998.
[8] M. Bernstein, G. Little, R. Miller., B. Hartmann, M. Ackerman., D. Karger, D. Crowell,, and K. Panovich., Soylent: a word processor with a crowd inside. In Proc. of the 23rd symposium on User interface software and technology, pages 313-322. ACM, 2010.
[9] E. Bertini and D. Lalanne, Investigating and reflecting on the integration of automatic data analysis and visualization in knowledge discovery ACM SIGKDD Explorations Newsletter, 11(2): 9-18, 2010.
[10] J. Bigham, C. Jayant, H. Ji., G. Little, A. Miller., R. Miller, R. Miller., A. Tatarowicz, B. White., S. White et al. Vizwiz: nearly real-time answers to visual questions. In , Proc. of the 23rd symposium on User interface software and technology, pages 333-342. ACM, 2010.
[11] F. Boujarwah, J. Kim, G. Abowd,, and R. Arriaga., Developing scripts to teach social skills: Can the crowd assist the author? In Workshops at the Twenty-Fifth AAAI Conf. on Artificial Intelligence, 2011.
[12] I. Bowman, S. Joshi, and J. Van, Horn. Query-based coordinated multiple views with feature similarity space for visual analysis of mri repositories. In Visual Analytics Science and Technology (VAST), Conf. on, pages 267-268. IEEE, 2011.
[13] M. Chen, D. Ebert, H. Hagen., R. Laramee, R. Van Liere, K. Ma, W. Ribarsky., G. Scheuermann, and D. Silver, Data, information, and knowledge in visualization Computer Graphics and Applications, IEEE, 29(1): 12-19, 2009.
[14] Y. Chen, J. Alsakran, S. Barlowe., J. Yang, and Y. Zhao., Supporting effective common ground construction in asynchronous collaborative visual analytics. In Visual Analytics Science and Technology (VAST), Conf. on, pages 101-110. IEEE, 2011.
[15] Y. Chen, S. Barlowe, and J. Yang., Click2annotate: Automated insight externalization with rich semantics. In Visual Analytics Science and Technology (VAST), Symposium on, pages 155-162. IEEE, 2010.
[16] Y. Chen, B. Liem, and H. Zhang., An iterative dual pathway structure for speech-to-text transcription. In Human Computation: Papers from the AAAI Workshop (WS-11–11). San Francisco, CA, 2011.
[17] S. Chernova, J. Orkin, and C. Breazeal., Crowdsourcing hri through online multiplayer games. In Proc. Dialog with Robots: AAAI fall symposium, 2010.
[18] J. Choo, H. Lee, J. Kihm,, and H. Park., ivisclassifier: An interactive visual analytics system for classification based on supervised dimension reduction. In Visual Analytics Science and Technology (VAST), Symposium on, pages 27-34. IEEE, 2010.
[19] K. Chung, C. Chiu, X. Xiao,, and P. Chi., Stress outsourced: a haptic social network via crowdsourcing. In Proc. of the 27th SIGCHI Conf. on Human factors in computing systems, pages 2439-2448. ACM, 2009.
[20] S. Cooper, F. Khatib, A. Treuille., J. Barbero, J. Lee., M. Beenen, A. Leaver-Fay, D. Baker, Z. Popovic, et al. Predicting protein structures with a multiplayer online game. Nature, 466(7307): 756-760, 2010.
[21] B. Da Silva, and A. Garcia., Ka-captcha: An opportunity for knowledge acquisition on the web. In Proc. of the national Conf. on Artificial Intelligence, 22, page 1322, 2007.
[22] S. Dekker and D. Woods, Maba-maba or abracadabra? progress on human-automation coordination Cognition, Technology & Work, 4(4): 240-244. 2002.
[23] W. Dou, X. Wang, R. Chang,, and W. Ribarsky., Paralleltopics: A probabilistic approach to exploring document collections. Visual Analytics Science and Technology (VAST), Conf. on, 2011.
[24] P. Dourish., Where the action is: the foundations of embodied interaction. The MIT Press, 2004.
[25] H. Fastl and E. Zwicker., Psychoacoustics: facts and models, 22. Springer-Verlag New York Inc, 2007.
[26] G. Ferguson and J. Allen., Trips: An integrated intelligent problem-solving assistant. In Proc. of the National Conf. on Artificial Intelligence, pages 567-573. JOHN WILEY & SONS LTD, 1998.
[27] P. Fitts., Human engineering for an effective air-navigation and traffic-control system. 1951.
[28] R. Fuchs, J. Waser, and M. Groller, Visual human+ machine learning Visualization and Computer Graphics, Transactions on, 15(6): 1327-1334, 2009.
[29] S. Garg,1. Ramakrishnan,, and K. Mueller., A visual analytics approach to model learning. In Visual Analytics Science and Technology (VAST), Symposium on, pages 67-74. IEEE, 2010.
[30] W. Gaver., Technology affordances. In Proc. of the SIGCHI Conf. on Human factors in computing systems: Reaching through technology, pages 79-84. ACM. 1991.
[31] J. Gibson, The theory of affordances Perceiving, acting, and knowing, pages 67-82, 1977.
[32] J. Gibson., The ecological approach to visual perception. Lawrence Erlbaum. 1986.
[33] C. Ho, T. Chang, J. Lee., J. Hsu, and K. Chen., Kisskissban: a competitive human computation game for image annotation. In Proc. of the SIGKDD Workshop on Human Computation, pages 11-14. ACM, 2009.
[34] E. Horvitz., Principles of mixed-initiative user interfaces. In Proc. of the SIGCHI Conf. on Human factors in computing systems: the CHI is the limit, pages 159-166. ACM, 1999.
[35] C. Hu, B. Bederson, P. Resnik,, and Y. Kronrod., Monotrans2: A new human computation system to support monolingual translation. In Proc. of the 29th SIGCHI Conf. on Human factors in computing systems, pages 1133-1136. ACM, 2011.
[36] S. Ingram, T. Munzner, V. Irvine., M. Tory, S. Bergner,, and T. Mller., Dimstiller: Workflows for dimensional analysis and reduction. In Visual Analytics Science and Technology (VAST), Symposium on, pages 3-10. IEEE, 2010.
[37] C. Ip and A. Varshney, Saliency-assisted navigation of very large land-scape images Visualization and Computer Graphics, Transactions on, 17(12): 1737-1746, 2011.
[38] N. Jordan, Allocation of functions between man and machines in automated systems Journal of applied psychology, 47(3): 161, 1963.
[39] N. Kadivar, V. Chen, D. Dunsmuir., E. Lee, C. Qian., J. Dill, C. Shaw,, and R. Woodbury., Capturing and supporting the analysis process. In Visual Analytics Science and Technology (VAST), Symposium on, pages 131-138. IEEE, 2009.
[40] S. Kandel, A. Paepcke, J. Hellerstein,, and J. Heer., Wrangler: Interactive visual specification of data transformation scripts. In Proc. of the 2011 Conf. on Human factors in computing systems, pages 3363-3372. ACM, 2011.
[41] S. Koch, H. Bosch, M. Giereth,, and T. Ertl., Iterative integration of visual insights during patent search and analysis. In Visual Analytics Science and Technologv (VAST), Svmposium on. pages 203-210. IEEE. 2009.
[42] A. Kulkarni., The complexity of crowdsourcing: Theoretical problems in human computation. In CHI Workshop on Crowdsourcing and Human Computation. 2011.
[43] Y. Kuo, J. Lee, K. Chiang., R. Wang, E. Shen., C. Chan, and J. Hsu., Community-based game design: experiments on social games for commonsense data collection. In Proc. of the ACM SIGKDD Workshop on Human Computation, pages 15-22. ACM, 2009.
[44] E. Law,L. Von Ahn, R. Dannenberg, and M. Crawford, Tagatune: A game for music and sound annotation Proc. of ISMIR (Vienna, Austria), 2007.
[45] H. Lieberman, D. Smith, and A. Teeters., Common consensus: a web-based game for collecting commonsense goals. In ACM Workshop on Common Sense for Intelligent Interfaces, 2007.
[46] H. Lipford, F. Stukes, W. Dou., M. Hawkins, and R. Chang., Helping users recall their reasoning process. In Visual Analytics Science and Technology (VAST), Symposium on, pages 187-194. IEEE, 2010.
[47] Z. Liu, B. Lee, S. Kandula,, and R. Mahajan., Netclinic: Interactive visualization to enhance automated fault diagnosis in enterprise networks. In Visual Analytics Science and Technology (VAST), Symposium on, pages 131-138. IEEE. 2010.
[48] H. Ma, R. Chandrasekar, C. Quirk,, and A. Gupta., Page hunt: using human computation games to improve web search. In Proc. of the ACM SIGKDD Workshop on Human Computation. pages 27-28. ACM. 2009.
[49] M. Mandel and D. Ellis., A web-based game for collecting music meta-data. Journal of New Music Research. 37(2): 151165. 2008.
[50] M. Migut,J. van Gernert,, and M. Worring., Interactive decision making using dissimilarity to visually represented prototypes. In Visual Analytics Science and Technologv (VAST), Conf. on. pages 141-149. IEEE. 2011.
[51] M. Migut and M. Worring., Visual exploration of classification models for risk assessment. In Visual Analytics Science and Technology (VAST), Symposium on, pages 11-18. IEEE, 2010.
[52] G. Moore et al. Cramming more components onto integrated circuits. Proc. of the IEEE, 86(1): 82-85, 1998.
[53] D. Norman., The design of everyday things. Basic books, 2002.
[54] H. Price, The allocation of functions in systems Human Factors: The Journal of the Human Factors and Ergonomics Society, 27(1): 33-45, 1985.
[55] A. Quinn and B. Bederson., Human computation: a survey and taxonomy of a growing field. In Proc. of the 29th SIGCHI Conf. on Human factors in computing systems, pages 1403-1412. ACM, 2011.
[56] J. Rickel and W. Johnson, Animated agents for procedural training in virtual reality: Perception, cognition, and motor control Applied Artificial Intelligence, 13(4-5): 343-382, 1999.
[57] M. Ruiz, A. Bardera, I. Boada,, and I. Viola., Automatic transfer functions based on informational divergence. Visualization and Computer Graphics, Transactions on, 17(12): 1932-1941, 2011.
[58] M. Runco., Creativity: Theories and themes: Research, development, and practice. Academic Press, 2007.
[59] B. Russell, A. Torralba, K. Murphy,, and W. Freeman., Labelme: a database and web-based tool for image annotation. Int'l journal of computer vision, 77(1): 157-173, 2008.
[60] P. Shah and A. Miyake., The Cambridge handbook of visuospatial thinking. Cambridge Univ Pr, 2005.
[61] D. Shahaf and E. Amir., Towards a theory of ai completeness. In 8th Interational symposium on logic formalizations of commonsense reasoning, 2007.
[62] P. Shenoy and D. Tan., Human-aided computing: Utilizing implicit human processing to classify images. In Proc. of the 26th SIGCHI Conf. on Humanfactors in computing systems, pages 845-854. ACM, 2008.
[63] T. Sheridan., Function allocation: algorithm, alchemy or apostasy? Int’ l Journal of Human-Computer Studies, 52(2): 203-216, 2000.
[64] B. Shneiderman., The eyes have it: A task by data type taxonomy for information visualizations. In Visual Languages, 1996. Proc., IEEE Symposium on, pages 336-343. IEEE, 1996.
[65] Y. Shrinivasan, D. Gotzy, and J. Lu., Connecting the dots in visual analysis. In Visual Analytics Science and Technology (VAST), Symposium on, pages 123-130. IEEE, 2009.
[66] C. Steed, J. Swan, T. Jankun-Kelly,, and P. Fitzpatrick., Guided analysis of hurricane trends using statistical processes integrated with interactive parallel coordinates. In Visual Analytics Science and Technology (VAST), Symposium on, pages 19-26. IEEE, 2009.
[67] Y. Tanaka, Y. Sakamoto, and T. Kusumi., Conceptual combination versus critical combination: Devising creative solutions using the sequential application of crowds. In Proc. of the 33rd Conf. of the Cognitive Science Society. 2011.
[68] A. Tatu, G. Albuquerque, M. Eisemann., J. Schneidewind, H. Theisel., M. Magnork, and D. Keim., Combining automated analysis and visualization techniques for effective exploration of high-dimensional data. In Visual Analytics Science and Technology (VAST), Symposium on, pages 59-66. IEEE. 2009.
[69] L. Terveen, Overview of human-computer collaboration Knowledge-Based Systems, 8(2-3): 67-81, 1995.
[70] J. Thomas and K. Cook., Illuminating the path: The research and development agenda for visual analytics, 54. IEEE, 2005.
[71] R. Valdés-Pérez., Principles of human-computer collaboration for knowledge discovery in science. Artificial Intelligence, 107(2): 335-346, 1999.
[72] L. Von, Ahn. Human Computation. PhD thesis, Carnegie Mellon University, 2005.
[73] L. Von Ahn and L. Dabbish., Labeling images with a computer game. In Proc. of the 22nd SIGCHI Conf. on Human factors in computing systems, pages 319-326. ACM, 2004.
[74] L. Von Ahn, S. Ginosar, M. Kedia., R. Liu, and M. Blum., Improving accessibility of the web with a computer game. In Proc. of the 24th SIGCHI Con! on Human factors in computing systems, pages 79-82. ACM, 2006.
[75] L. Von Ahn, M. Kedia, and M. Blum., Verbosity: a game for collecting commonsense facts. In Proc. of the 24th SIGCHI Conf. on Human factors in computing systems, pages 75-78. ACM, 2006.
[76] L. Von Ahn, R. Liu, and M. Blum., Peekaboom: a game for locating objects in images. In Proc. of the 24th SIGCHI Conf. on Human factors in computing systems, pages 55-64. ACM, 2006.
[77] L. Von Ahn, B. Maurer, C. McMillen., D. Abraham, and M. Blum, re-captcha: Human-based character recognition via web security measures Science, 321(5895): 1465-1468, 2008.
[78] L. Vygotsky., Thought and word. 1962.
[79] J. Wing, Five deep questions in computing Communications of the ACM, 51(1): 58-60, 2008.
[80] T. Yan, V. Kumar, and D. Ganesan., Crowdsearch: exploiting crowds for accurate real-time image search on mobile phones. In Proc. of the 8th Int'l Conf. on Mobile systems, applications, and services, pages 77-90. ACM, 2010.
[81] L. Yu and J. Nickerson., Cooks or cobblers?: crowd creativity through combination. In Proc. of the 29th SIGCHI Conf. on Human factors in computing systems, pages 1393-1402. ACM, 2011.
[82] M. Yuen, L. Chen, and I. King., A survey of human computation systems. In Computational Science and Engineering, Int'l Conf. on, 4, pages 723-728. IEEE, 2009.
[83] Z. Zheng, N. Ahmed, and K. Mueller, iview: A feature clustering framework for suggesting informative views in volume visualization Visualization and Computer Graphics, Transactions on, 17(12): 1959-1968, 2011.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool